2024-04-26T20:12:31.275 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:12:31.276 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/orig.config.yaml 2024-04-26T20:12:31.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:12:31.434 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} email: null first_in_suite: false job_id: '7675164' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Reduced data availability - Degraded data redundancy sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOu3G4ORMpFFzdyVlekiwhZkIWNMgClxKe2xZr7nLLokggQyFpD3KjXtfiCzRw12iWT0SOiI6I435VB5xx43s8k= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPeG7ialqsj8nIaFb16MqFvM/lqPQZj8cA0W+i2UkjT76rNJuy6/e7d9HlvZuJQyWgpzLSEQ9Ym86BwGF805KHk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:12:31.435 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:12:31.435 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:12:31.435 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:12:31.437 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:12:31.461 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:12:31.461 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:12:31.461 INFO:teuthology.packaging:ref: None 2024-04-26T20:12:31.461 INFO:teuthology.packaging:tag: None 2024-04-26T20:12:31.461 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:12:31.461 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:12:31.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:12:31.596 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:12:31.598 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:12:31.604 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:12:31.604 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:12:31.610 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:12:31.627 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:12:31.633 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:12:31.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:07:35.582378', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOu3G4ORMpFFzdyVlekiwhZkIWNMgClxKe2xZr7nLLokggQyFpD3KjXtfiCzRw12iWT0SOiI6I435VB5xx43s8k='} 2024-04-26T20:12:31.684 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:07:35.580888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPeG7ialqsj8nIaFb16MqFvM/lqPQZj8cA0W+i2UkjT76rNJuy6/e7d9HlvZuJQyWgpzLSEQ9Ym86BwGF805KHk='} 2024-04-26T20:12:31.684 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:12:31.691 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T20:12:31.691 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T20:12:31.691 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:12:31.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f38c2b43160>, signals=[15]) 2024-04-26T20:12:31.762 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:12:31.768 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:12:31.768 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:12:31.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:12:31.850 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:12:31.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:12:31.927 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:12:31.932 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-04-26T20:12:31.951 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-04-26T20:12:31.951 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:12:32.010 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:12:32.011 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:12:32.012 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-04-26T20:12:32.036 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-26T20:12:32.053 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-26T20:12:32.053 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-26T20:12:32.110 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-26T20:12:32.110 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-26T20:12:32.110 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-26T20:12:32.110 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:12:32.111 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:12:32.112 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-26T20:12:32.136 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:12:32.143 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:12:32.148 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:12:32.148 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:12:32.150 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:12:32.167 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:12:32.173 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:12:32.173 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:12:32.208 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:12:32.309 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:12:32.361 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:12:32.375 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:12:32.375 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T20:12:32.375 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:12:32.375 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:12:32.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:12:32.393 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.393 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:12:32.393 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-26T20:12:32.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:12:32.394 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.394 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:12:32.394 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-04-26T20:12:32.450 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:12:32.451 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.451 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:12:32.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.455 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:12:32.456 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.456 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:12:32.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:12:32.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum upgrade -y linux-firmware 2024-04-26T20:12:32.604 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum upgrade -y linux-firmware 2024-04-26T20:12:34.932 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T20:12:35.130 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-26T20:12:39.389 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-26T20:12:43.831 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-26T20:12:45.441 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-26T20:12:57.014 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 928 kB/s | 19 MB 00:20 2024-04-26T20:12:58.239 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 21 MB 00:12 2024-04-26T20:13:03.381 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T20:13:03.456 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.2 MB 00:02 2024-04-26T20:13:05.349 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-26T20:13:05.350 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-26T20:13:05.350 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-26T20:13:05.463 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:13:05.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:13:05.610 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:13:05.610 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:13:05.612 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi164.front.sepia.ceph.com to complete... 2024-04-26T20:13:07.732 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 5.3 kB/s | 16 kB 00:03 2024-04-26T20:13:12.327 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 21 MB 00:04 2024-04-26T20:13:17.641 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T20:13:19.702 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-26T20:13:19.703 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-04-26T20:13:19.704 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-26T20:13:19.808 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:13:19.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:13:20.084 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:13:20.084 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:13:20.086 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi043.front.sepia.ceph.com to complete... 2024-04-26T20:13:26.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26 20:13:26 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:13:26.272 INFO:teuthology.orchestra.run.smithi164.stderr:FINISHED --2024-04-26 20:13:26-- 2024-04-26T20:13:26.272 INFO:teuthology.orchestra.run.smithi164.stderr:Total wall clock time: 21s 2024-04-26T20:13:26.272 INFO:teuthology.orchestra.run.smithi164.stderr:Downloaded: 1 files, 341M in 20s (16.7 MB/s) 2024-04-26T20:13:26.275 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:13:26.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:13:28.362 INFO:teuthology.orchestra.run.smithi164.stdout:Verifying... ######################################## 2024-04-26T20:13:28.533 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing... ######################################## 2024-04-26T20:13:28.635 INFO:teuthology.orchestra.run.smithi164.stdout:Updating / installing... 2024-04-26T20:13:36.554 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:14:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26 20:14:00 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:14:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:FINISHED --2024-04-26 20:14:00-- 2024-04-26T20:14:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:Total wall clock time: 41s 2024-04-26T20:14:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:Downloaded: 1 files, 341M in 40s (8.59 MB/s) 2024-04-26T20:14:00.804 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:14:00.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:14:02.982 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying... ######################################## 2024-04-26T20:14:03.120 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing... ######################################## 2024-04-26T20:14:03.238 INFO:teuthology.orchestra.run.smithi043.stdout:Updating / installing... 2024-04-26T20:14:11.515 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:14:15.534 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:14:15.534 DEBUG:teuthology.orchestra.run.smithi164:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:14:15.610 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:14:15.611 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:14:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:14:15.613 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:14:15.614 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:14:15.615 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:14:15.616 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:14:15.617 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:14:15.618 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:14:15.619 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:14:15.620 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:14:15.621 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:14:15.622 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:14:15.623 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:14:15.624 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:14:15.625 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:14:15.626 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:14:15.627 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:14:15.628 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:14:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:14:15.630 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:14:15.631 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:14:15.632 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:14:15.633 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:14:15.634 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:14:15.635 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:14:15.636 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:14:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:14:15.638 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:14:15.639 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:14:15.640 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:14:15.641 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:14:15.642 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:14:15.643 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:14:15.645 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:14:15.646 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:14:15.647 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:14:15.648 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:14:15.649 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:14:15.650 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:14:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:14:15.652 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:14:15.653 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:14:15.654 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:14:15.655 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:14:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:14:15.657 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:14:15.658 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:14:15.659 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:14:15.660 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:14:15.661 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:14:15.662 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:14:15.663 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:14:15.664 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:14:15.665 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:14:15.666 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:14:15.667 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:14:15.668 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:14:15.669 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:14:15.670 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:14:15.671 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:14:15.672 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:14:15.673 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:14:15.674 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:14:15.675 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:14:15.676 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:14:15.677 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:14:15.678 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:14:15.679 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:14:15.680 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:14:15.681 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:14:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:14:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:14:15.684 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:14:15.685 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:14:15.687 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:14:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:14:15.689 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:14:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:14:15.691 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:14:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:14:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:14:15.694 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:14:15.695 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:14:15.696 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:14:15.697 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:14:15.698 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:14:15.699 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:14:15.700 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:14:15.701 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:14:15.702 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:14:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:14:15.704 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:14:15.705 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:14:15.706 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:14:15.707 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:14:15.709 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:14:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:14:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:14:15.712 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:14:15.713 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:14:15.714 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:14:15.715 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:14:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:14:15.717 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:14:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:14:15.719 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:14:15.720 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:14:15.721 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:14:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:14:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:14:15.724 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:14:15.725 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:14:15.726 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:14:15.727 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:14:15.728 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:14:15.729 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:14:15.730 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:14:15.731 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:14:15.732 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:14:15.733 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:14:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:14:15.735 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:14:15.736 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:14:15.737 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:14:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:14:15.739 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:14:15.740 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:14:15.741 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:14:15.742 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:14:15.743 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:14:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:14:15.745 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:14:15.746 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:14:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:14:15.748 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:14:15.749 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:14:15.750 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:14:15.751 INFO:teuthology.orchestra.run.smithi164.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:14:15.755 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.755 DEBUG:teuthology.orchestra.run.smithi164:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:14:15.777 INFO:teuthology.orchestra.run.smithi164.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:14:15.777 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:14:15.777 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:14:15.777 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: fi 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T20:14:15.778 INFO:teuthology.orchestra.run.smithi164.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:14:15.779 INFO:teuthology.orchestra.run.smithi164.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:14:15.779 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.779 INFO:teuthology.orchestra.run.smithi164.stdout:fi 2024-04-26T20:14:15.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-04-26T20:14:15.856 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 68.el9 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8264254 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:14:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:14:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:14:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:14:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:14:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-04-26T20:14:15.859 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:14:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:14:15.860 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:14:15.860 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:15.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:14:16.486 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-04-26T20:14:17.858 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:14:17.885 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-04-26T20:14:17.887 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-26T20:14:17.904 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.7ra05WIfeX 2024-04-26T20:14:17.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7ra05WIfeX 2024-04-26T20:14:17.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.7ra05WIfeX 2024-04-26T20:14:18.103 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.7ra05WIfeX is 6KB 2024-04-26T20:14:18.152 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.7ra05WIfeX 2024-04-26T20:14:18.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:14:18.228 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:14:18.228 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:14:18.228 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:14:18.228 INFO:teuthology.orchestra.run.smithi164.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:14:18.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:18.392 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-04-26T20:14:48.395 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T20:14:48.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:14:48.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:14:49.996 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:14:49.996 DEBUG:teuthology.orchestra.run.smithi043:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:14:50.122 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:14:50.123 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:14:50.124 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:14:50.125 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:14:50.126 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:14:50.127 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:14:50.128 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:14:50.129 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:14:50.130 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:14:50.131 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:14:50.132 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:14:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:14:50.134 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:14:50.135 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:14:50.136 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:14:50.137 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:14:50.138 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:14:50.139 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:14:50.140 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:14:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:14:50.142 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:14:50.143 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:14:50.144 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:14:50.145 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:14:50.147 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:14:50.148 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:14:50.149 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:14:50.150 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:14:50.151 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:14:50.152 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:14:50.153 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:14:50.154 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:14:50.155 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:14:50.156 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:14:50.157 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:14:50.158 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:14:50.159 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:14:50.160 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:14:50.161 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:14:50.162 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:14:50.163 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:14:50.164 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:14:50.165 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:14:50.166 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:14:50.167 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:14:50.168 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:14:50.169 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:14:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:14:50.171 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:14:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:14:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:14:50.174 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:14:50.175 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:14:50.176 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:14:50.177 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:14:50.178 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:14:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:14:50.180 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:14:50.181 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:14:50.182 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:14:50.183 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:14:50.184 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:14:50.185 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:14:50.186 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:14:50.187 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:14:50.188 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:14:50.189 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:14:50.190 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:14:50.191 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:14:50.192 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:14:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:14:50.194 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:14:50.195 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:14:50.196 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:14:50.197 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:14:50.198 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:14:50.199 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:14:50.200 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:14:50.201 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:14:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:14:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:14:50.204 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:14:50.205 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:14:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:14:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:14:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:14:50.209 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:14:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:14:50.211 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:14:50.212 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:14:50.213 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:14:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:14:50.215 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:14:50.216 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:14:50.217 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:14:50.218 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:14:50.219 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:14:50.220 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:14:50.221 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:14:50.222 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:14:50.223 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:14:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:14:50.225 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:14:50.226 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:14:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:14:50.228 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:14:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:14:50.230 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:14:50.231 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:14:50.232 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:14:50.233 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:14:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:14:50.235 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:14:50.236 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:14:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:14:50.238 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:14:50.239 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:14:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:14:50.241 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:14:50.242 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:14:50.243 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:14:50.244 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:14:50.245 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:14:50.246 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:14:50.247 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:14:50.248 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:14:50.249 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:14:50.250 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:14:50.251 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:14:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:14:50.253 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:14:50.254 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:14:50.255 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:14:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:14:50.257 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:14:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:14:50.259 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:14:50.263 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.263 DEBUG:teuthology.orchestra.run.smithi043:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: fi 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.290 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-04-26T20:14:50.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 68.el9 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8264254 2024-04-26T20:14:50.372 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:14:50.373 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-04-26T20:14:50.374 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:14:50.374 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:14:50.374 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:14:50.375 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:50.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:14:50.919 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-04-26T20:14:52.409 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:14:52.423 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-04-26T20:14:52.426 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-04-26T20:14:52.468 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.5aMKaA48KH 2024-04-26T20:14:52.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5aMKaA48KH 2024-04-26T20:14:52.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.5aMKaA48KH 2024-04-26T20:14:52.676 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.5aMKaA48KH is 6KB 2024-04-26T20:14:52.697 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.5aMKaA48KH 2024-04-26T20:14:52.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:14:52.782 INFO:teuthology.orchestra.run.smithi043.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:14:52.782 INFO:teuthology.orchestra.run.smithi043.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:14:52.782 INFO:teuthology.orchestra.run.smithi043.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:14:52.782 INFO:teuthology.orchestra.run.smithi043.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:14:52.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:14:52.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-04-26T20:15:06.786 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T20:15:15.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:15:15.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:15:18.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T20:15:22.958 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T20:15:22.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:15:22.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:15:30.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:15:30.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:15:41.343 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-04-26T20:15:49.283 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-04-26T20:15:50.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:15:50.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:16:04.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:16:04.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:16:04.778 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-26T20:16:05.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:16:05.357 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:16:05.357 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-26T20:16:05.394 INFO:teuthology.orchestra.run.smithi164.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:16:05.395 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:16:05.395 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:16:05.395 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:16:06.396 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T20:16:06.396 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:16:06.533 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-04-26T20:16:06.582 DEBUG:teuthology.parallel:result is None 2024-04-26T20:16:50.349 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:17:02.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:17:02.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:17:02.809 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-26T20:17:03.478 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:17:03.478 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:17:03.478 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-04-26T20:17:03.532 INFO:teuthology.orchestra.run.smithi043.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:17:03.532 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:17:03.532 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:17:03.532 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:17:04.533 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T20:17:04.533 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:17:04.681 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-04-26T20:17:04.730 DEBUG:teuthology.parallel:result is None 2024-04-26T20:17:04.731 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:17:04.738 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:17:04.738 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:17:04.745 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:17:04.802 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:17:04.817 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:17:04.823 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:17:04.824 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:17:04.831 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:17:04.923 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:17:04.929 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:17:04.930 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:17:04.937 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:17:05.029 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:17:05.032 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:17:05.061 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:17:05.062 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:17:05.069 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:17:05.076 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:17:05.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:17:05.082 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:17:05.167 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:17:05.177 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:17:05.177 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:17:05.184 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:17:05.237 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:17:05.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:17:05.419 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:17:05.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:17:05.603 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:17:05.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:17:05.691 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:17:05.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:17:05.878 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:17:05.962 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:17:06.048 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:17:06.048 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:17:06.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-04-26T20:17:06.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-26T20:17:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:17:06.293 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:17:06.784 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:17:06.791 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:17:06.791 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:17:06.802 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:17:06.811 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:17:06.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-04-26T20:17:06.916 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-04-26T20:17:06.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-26T20:17:07.063 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-26T20:17:07.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:17:07.069 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-04-26T20:17:07.140 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-04-26T20:17:07.140 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-26T20:17:07.193 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-26T20:17:07.193 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:17:07.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:17:07.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:17:07.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:17:07.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:17:07.349 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:17:07.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:17:07.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:17:07.505 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:17:07.516 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:17:07.534 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:17:07.535 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T20:22:20.827 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-26T20:22:20.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:22:20.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:22:20.950 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-26T20:22:21.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-26T20:22:21.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:22:21.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:22:21.188 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-26T20:22:21.345 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-26T20:22:21.345 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T20:22:21.355 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T20:22:21.355 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:22:21.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:22:21.363 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:22:21.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:22:21.449 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:22:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:22:21.491 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:22:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:22:21.562 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-26T20:22:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-04-26T20:22:21.592 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-26T20:22:21.610 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-04-26T20:22:21.629 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:22:21.655 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:22:21.667 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:22:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:22:21.805 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:22:21.814 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:22:21.814 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-26T20:22:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:22:21.859 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:22:21.859 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-26T20:22:21.859 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.860 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.860 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.860 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:22:21.861 INFO:teuthology.run_tasks:Running task install... 2024-04-26T20:22:21.868 DEBUG:teuthology.task.install:project ceph 2024-04-26T20:22:21.869 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T20:22:21.869 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T20:22:21.869 INFO:teuthology.task.install:Using flavor: default 2024-04-26T20:22:21.875 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T20:22:21.875 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T20:22:21.875 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:22:21.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:22:21.878 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:22:21.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:22:22.021 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:22:22.021 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:22:22.093 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:22:22.093 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:22:22.194 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:22:22.195 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:22:22.195 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:22:22.202 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:22:22.202 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:22:22.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:22:22.287 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:22:22.287 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:22:22.291 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:22:22.292 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:22:22.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:22:22.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:22:22.527 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:22:22.550 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:22:22.604 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-26T20:22:22.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-26T20:22:22.638 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-04-26T20:22:22.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-04-26T20:22:22.913 INFO:teuthology.orchestra.run.smithi164.stdout:48 files removed 2024-04-26T20:22:22.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:22:22.982 INFO:teuthology.orchestra.run.smithi043.stdout:48 files removed 2024-04-26T20:22:23.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:22:23.780 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 213 kB/s | 84 kB 00:00 2024-04-26T20:22:23.896 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 199 kB/s | 84 kB 00:00 2024-04-26T20:22:24.280 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-26T20:22:24.496 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-26T20:22:24.789 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-26T20:22:24.946 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-26T20:22:25.479 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T20:22:26.794 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.0 MB 00:01 2024-04-26T20:22:28.030 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-26T20:22:30.879 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-04-26T20:22:33.348 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-26T20:22:35.123 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-26T20:22:36.063 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-26T20:22:36.217 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-26T20:22:37.688 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-26T20:22:39.579 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T20:22:42.133 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T20:22:44.534 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:22:44.535 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:22:44.556 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:22:44.558 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:22:44.559 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:22:44.560 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:22:44.563 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:22:44.566 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:22:44.567 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:22:44.638 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:22:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:22:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:22:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:22:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:22:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:22:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:22:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:22:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:22:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:22:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:22:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 4 Packages 2024-04-26T20:22:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:22:44.671 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 222 M 2024-04-26T20:22:44.671 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-26T20:22:45.451 INFO:teuthology.orchestra.run.smithi164.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-26T20:22:45.693 INFO:teuthology.orchestra.run.smithi164.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.6 MB/s | 889 kB 00:00 2024-04-26T20:22:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T20:22:45.776 INFO:teuthology.orchestra.run.smithi164.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.7 MB/s | 145 kB 00:00 2024-04-26T20:22:45.935 INFO:teuthology.orchestra.run.smithi164.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 7.3 MB/s | 5.4 MB 00:00 2024-04-26T20:22:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-04-26T20:22:46.244 INFO:teuthology.orchestra.run.smithi164.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-04-26T20:22:47.136 INFO:teuthology.orchestra.run.smithi164.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.0 MB/s | 4.7 MB 00:00 2024-04-26T20:22:47.378 INFO:teuthology.orchestra.run.smithi164.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 9.7 MB/s | 21 MB 00:02 2024-04-26T20:22:47.437 INFO:teuthology.orchestra.run.smithi164.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 441 kB/s | 25 kB 00:00 2024-04-26T20:22:48.087 INFO:teuthology.orchestra.run.smithi164.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 9.1 MB/s | 17 MB 00:01 2024-04-26T20:22:48.179 INFO:teuthology.orchestra.run.smithi164.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.4 MB/s | 225 kB 00:00 2024-04-26T20:22:48.218 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:22:48.219 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:22:48.241 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:22:48.243 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:22:48.244 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:22:48.245 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:22:48.248 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:22:48.252 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:22:48.253 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:22:48.279 INFO:teuthology.orchestra.run.smithi164.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 272 kB/s | 27 kB 00:00 2024-04-26T20:22:48.329 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-26T20:22:48.349 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:22:48.350 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:22:48.351 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:22:48.352 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:22:48.353 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:22:48.354 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:22:48.355 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:22:48.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:22:48.357 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:22:48.358 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:22:48.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:22:48.360 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 4 Packages 2024-04-26T20:22:48.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:22:48.362 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 222 M 2024-04-26T20:22:48.362 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-26T20:22:48.387 INFO:teuthology.orchestra.run.smithi164.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 295 kB/s | 32 kB 00:00 2024-04-26T20:22:48.629 INFO:teuthology.orchestra.run.smithi164.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 747 kB 00:00 2024-04-26T20:22:48.738 INFO:teuthology.orchestra.run.smithi164.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 163 kB 00:00 2024-04-26T20:22:48.813 INFO:teuthology.orchestra.run.smithi164.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T20:22:49.022 INFO:teuthology.orchestra.run.smithi164.stdout:(17/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 5.6 MB/s | 11 MB 00:01 2024-04-26T20:22:49.063 INFO:teuthology.orchestra.run.smithi164.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.0 MB/s | 501 kB 00:00 2024-04-26T20:22:49.122 INFO:teuthology.orchestra.run.smithi164.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 783 kB/s | 45 kB 00:00 2024-04-26T20:22:49.189 INFO:teuthology.orchestra.run.smithi164.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T20:22:49.265 INFO:teuthology.orchestra.run.smithi164.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T20:22:49.405 INFO:teuthology.orchestra.run.smithi164.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.3 MB/s | 321 kB 00:00 2024-04-26T20:22:49.515 INFO:teuthology.orchestra.run.smithi164.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 298 kB 00:00 2024-04-26T20:22:49.590 INFO:teuthology.orchestra.run.smithi164.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T20:22:49.657 INFO:teuthology.orchestra.run.smithi164.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T20:22:49.708 INFO:teuthology.orchestra.run.smithi043.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-26T20:22:49.766 INFO:teuthology.orchestra.run.smithi164.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-04-26T20:22:49.849 INFO:teuthology.orchestra.run.smithi164.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T20:22:49.916 INFO:teuthology.orchestra.run.smithi164.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 519 kB/s | 34 kB 00:00 2024-04-26T20:22:49.983 INFO:teuthology.orchestra.run.smithi164.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T20:22:50.133 INFO:teuthology.orchestra.run.smithi043.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 889 kB 00:00 2024-04-26T20:22:50.234 INFO:teuthology.orchestra.run.smithi043.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T20:22:50.525 INFO:teuthology.orchestra.run.smithi164.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.6 MB/s | 3.1 MB 00:00 2024-04-26T20:22:50.650 INFO:teuthology.orchestra.run.smithi164.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.3 MB/s | 3.5 MB 00:00 2024-04-26T20:22:50.751 INFO:teuthology.orchestra.run.smithi164.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.5 MB/s | 249 kB 00:00 2024-04-26T20:22:50.892 INFO:teuthology.orchestra.run.smithi043.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 3.6 MB/s | 2.4 MB 00:00 2024-04-26T20:22:51.134 INFO:teuthology.orchestra.run.smithi043.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-04-26T20:22:51.309 INFO:teuthology.orchestra.run.smithi164.stdout:(33/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T20:22:51.334 INFO:teuthology.orchestra.run.smithi164.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 85 kB/s | 49 kB 00:00 2024-04-26T20:22:51.376 INFO:teuthology.orchestra.run.smithi043.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-26T20:22:51.385 INFO:teuthology.orchestra.run.smithi164.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 221 kB/s | 16 kB 00:00 2024-04-26T20:22:51.468 INFO:teuthology.orchestra.run.smithi164.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 273 kB 00:00 2024-04-26T20:22:51.569 INFO:teuthology.orchestra.run.smithi164.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 183 kB/s | 18 kB 00:00 2024-04-26T20:22:51.627 INFO:teuthology.orchestra.run.smithi164.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-26T20:22:51.661 INFO:teuthology.orchestra.run.smithi164.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 885 kB/s | 80 kB 00:00 2024-04-26T20:22:51.753 INFO:teuthology.orchestra.run.smithi164.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-26T20:22:51.803 INFO:teuthology.orchestra.run.smithi164.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-26T20:22:51.826 INFO:teuthology.orchestra.run.smithi043.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.8 MB/s | 4.7 MB 00:00 2024-04-26T20:22:51.862 INFO:teuthology.orchestra.run.smithi164.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T20:22:51.929 INFO:teuthology.orchestra.run.smithi164.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T20:22:51.987 INFO:teuthology.orchestra.run.smithi164.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-26T20:22:52.046 INFO:teuthology.orchestra.run.smithi164.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-26T20:22:52.088 INFO:teuthology.orchestra.run.smithi164.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-26T20:22:52.138 INFO:teuthology.orchestra.run.smithi164.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T20:22:52.347 INFO:teuthology.orchestra.run.smithi164.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-26T20:22:52.397 INFO:teuthology.orchestra.run.smithi164.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-26T20:22:52.448 INFO:teuthology.orchestra.run.smithi164.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 436 kB/s | 21 kB 00:00 2024-04-26T20:22:52.481 INFO:teuthology.orchestra.run.smithi164.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 805 kB/s | 107 kB 00:00 2024-04-26T20:22:52.531 INFO:teuthology.orchestra.run.smithi164.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-26T20:22:52.582 INFO:teuthology.orchestra.run.smithi164.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 329 kB/s | 33 kB 00:00 2024-04-26T20:22:52.623 INFO:teuthology.orchestra.run.smithi164.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-26T20:22:52.665 INFO:teuthology.orchestra.run.smithi164.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 221 kB/s | 18 kB 00:00 2024-04-26T20:22:52.732 INFO:teuthology.orchestra.run.smithi164.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-26T20:22:52.774 INFO:teuthology.orchestra.run.smithi164.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-26T20:22:52.825 INFO:teuthology.orchestra.run.smithi164.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 497 kB/s | 45 kB 00:00 2024-04-26T20:22:52.933 INFO:teuthology.orchestra.run.smithi164.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-26T20:22:53.009 INFO:teuthology.orchestra.run.smithi164.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 343 kB/s | 25 kB 00:00 2024-04-26T20:22:53.018 INFO:teuthology.orchestra.run.smithi043.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 5.8 MB/s | 21 MB 00:03 2024-04-26T20:22:53.068 INFO:teuthology.orchestra.run.smithi043.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 516 kB/s | 25 kB 00:00 2024-04-26T20:22:53.125 INFO:teuthology.orchestra.run.smithi164.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-04-26T20:22:53.178 INFO:teuthology.orchestra.run.smithi164.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 281 kB/s | 47 kB 00:00 2024-04-26T20:22:53.209 INFO:teuthology.orchestra.run.smithi164.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.5 MB/s | 582 kB 00:00 2024-04-26T20:22:53.269 INFO:teuthology.orchestra.run.smithi043.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 7.3 MB/s | 11 MB 00:01 2024-04-26T20:22:53.284 INFO:teuthology.orchestra.run.smithi164.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 475 kB/s | 35 kB 00:00 2024-04-26T20:22:53.326 INFO:teuthology.orchestra.run.smithi164.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-04-26T20:22:53.360 INFO:teuthology.orchestra.run.smithi164.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-04-26T20:22:53.369 INFO:teuthology.orchestra.run.smithi043.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T20:22:53.444 INFO:teuthology.orchestra.run.smithi043.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 365 kB/s | 27 kB 00:00 2024-04-26T20:22:53.528 INFO:teuthology.orchestra.run.smithi043.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 388 kB/s | 32 kB 00:00 2024-04-26T20:22:53.762 INFO:teuthology.orchestra.run.smithi043.stdout:(14/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 7.1 MB/s | 17 MB 00:02 2024-04-26T20:22:53.820 INFO:teuthology.orchestra.run.smithi043.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.5 MB/s | 747 kB 00:00 2024-04-26T20:22:53.835 INFO:teuthology.orchestra.run.smithi164.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-04-26T20:22:53.853 INFO:teuthology.orchestra.run.smithi043.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 163 kB 00:00 2024-04-26T20:22:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T20:22:53.911 INFO:teuthology.orchestra.run.smithi164.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-26T20:22:53.988 INFO:teuthology.orchestra.run.smithi043.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 3.7 MB/s | 501 kB 00:00 2024-04-26T20:22:54.036 INFO:teuthology.orchestra.run.smithi164.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-26T20:22:54.071 INFO:teuthology.orchestra.run.smithi043.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 546 kB/s | 45 kB 00:00 2024-04-26T20:22:54.128 INFO:teuthology.orchestra.run.smithi164.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-26T20:22:54.163 INFO:teuthology.orchestra.run.smithi043.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.5 MB/s | 135 kB 00:00 2024-04-26T20:22:54.195 INFO:teuthology.orchestra.run.smithi164.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 2.5 MB/s | 911 kB 00:00 2024-04-26T20:22:54.284 INFO:teuthology.orchestra.run.smithi043.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.3 MB/s | 161 kB 00:00 2024-04-26T20:22:54.362 INFO:teuthology.orchestra.run.smithi164.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 528 kB/s | 87 kB 00:00 2024-04-26T20:22:54.389 INFO:teuthology.orchestra.run.smithi043.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.0 MB/s | 321 kB 00:00 2024-04-26T20:22:54.487 INFO:teuthology.orchestra.run.smithi164.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-04-26T20:22:54.506 INFO:teuthology.orchestra.run.smithi043.stdout:(23/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.7 MB/s | 5.4 MB 00:00 2024-04-26T20:22:54.540 INFO:teuthology.orchestra.run.smithi043.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 298 kB 00:00 2024-04-26T20:22:54.589 INFO:teuthology.orchestra.run.smithi043.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T20:22:54.621 INFO:teuthology.orchestra.run.smithi164.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-26T20:22:54.623 INFO:teuthology.orchestra.run.smithi043.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T20:22:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-26T20:22:54.779 INFO:teuthology.orchestra.run.smithi164.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-26T20:22:54.807 INFO:teuthology.orchestra.run.smithi043.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 415 kB/s | 34 kB 00:00 2024-04-26T20:22:54.888 INFO:teuthology.orchestra.run.smithi164.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 6.8 MB/s | 5.8 MB 00:00 2024-04-26T20:22:54.907 INFO:teuthology.orchestra.run.smithi043.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T20:22:54.929 INFO:teuthology.orchestra.run.smithi164.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 172 kB/s | 26 kB 00:00 2024-04-26T20:22:54.946 INFO:teuthology.orchestra.run.smithi164.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 333 kB/s | 19 kB 00:00 2024-04-26T20:22:54.991 INFO:teuthology.orchestra.run.smithi043.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.1 MB 00:00 2024-04-26T20:22:55.013 INFO:teuthology.orchestra.run.smithi164.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 529 kB/s | 35 kB 00:00 2024-04-26T20:22:55.039 INFO:teuthology.orchestra.run.smithi164.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-26T20:22:55.182 INFO:teuthology.orchestra.run.smithi164.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-26T20:22:55.323 INFO:teuthology.orchestra.run.smithi164.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-04-26T20:22:55.381 INFO:teuthology.orchestra.run.smithi164.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-26T20:22:55.415 INFO:teuthology.orchestra.run.smithi164.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 851 kB/s | 77 kB 00:00 2024-04-26T20:22:55.466 INFO:teuthology.orchestra.run.smithi043.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 6.3 MB/s | 3.5 MB 00:00 2024-04-26T20:22:55.523 INFO:teuthology.orchestra.run.smithi164.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-04-26T20:22:55.549 INFO:teuthology.orchestra.run.smithi164.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 254 kB/s | 42 kB 00:00 2024-04-26T20:22:55.575 INFO:teuthology.orchestra.run.smithi043.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.3 MB/s | 249 kB 00:00 2024-04-26T20:22:55.632 INFO:teuthology.orchestra.run.smithi164.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-26T20:22:55.667 INFO:teuthology.orchestra.run.smithi043.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 545 kB/s | 49 kB 00:00 2024-04-26T20:22:55.691 INFO:teuthology.orchestra.run.smithi164.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 979 kB/s | 269 kB 00:00 2024-04-26T20:22:55.724 INFO:teuthology.orchestra.run.smithi164.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T20:22:55.750 INFO:teuthology.orchestra.run.smithi164.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 439 kB/s | 51 kB 00:00 2024-04-26T20:22:55.759 INFO:teuthology.orchestra.run.smithi043.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 181 kB/s | 16 kB 00:00 2024-04-26T20:22:55.793 INFO:teuthology.orchestra.run.smithi164.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 539 kB/s | 54 kB 00:00 2024-04-26T20:22:55.850 INFO:teuthology.orchestra.run.smithi164.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 419 kB/s | 42 kB 00:00 2024-04-26T20:22:55.875 INFO:teuthology.orchestra.run.smithi164.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 535 kB/s | 44 kB 00:00 2024-04-26T20:22:55.884 INFO:teuthology.orchestra.run.smithi043.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 8.3 MB/s | 7.4 MB 00:00 2024-04-26T20:22:55.917 INFO:teuthology.orchestra.run.smithi043.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 273 kB 00:00 2024-04-26T20:22:55.934 INFO:teuthology.orchestra.run.smithi164.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 625 kB/s | 36 kB 00:00 2024-04-26T20:22:55.985 INFO:teuthology.orchestra.run.smithi043.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 7.5 MB/s | 762 kB 00:00 2024-04-26T20:22:56.009 INFO:teuthology.orchestra.run.smithi164.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 640 kB/s | 47 kB 00:00 2024-04-26T20:22:56.010 INFO:teuthology.orchestra.run.smithi043.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 199 kB/s | 18 kB 00:00 2024-04-26T20:22:56.034 INFO:teuthology.orchestra.run.smithi164.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-04-26T20:22:56.086 INFO:teuthology.orchestra.run.smithi043.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 808 kB/s | 80 kB 00:00 2024-04-26T20:22:56.110 INFO:teuthology.orchestra.run.smithi164.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 781 kB/s | 58 kB 00:00 2024-04-26T20:22:56.144 INFO:teuthology.orchestra.run.smithi043.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2024-04-26T20:22:56.186 INFO:teuthology.orchestra.run.smithi043.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T20:22:56.193 INFO:teuthology.orchestra.run.smithi164.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-26T20:22:56.210 INFO:teuthology.orchestra.run.smithi164.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 940 kB/s | 94 kB 00:00 2024-04-26T20:22:56.236 INFO:teuthology.orchestra.run.smithi043.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T20:22:56.269 INFO:teuthology.orchestra.run.smithi164.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 857 kB/s | 64 kB 00:00 2024-04-26T20:22:56.294 INFO:teuthology.orchestra.run.smithi164.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-04-26T20:22:56.311 INFO:teuthology.orchestra.run.smithi043.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.2 MB 00:00 2024-04-26T20:22:56.337 INFO:teuthology.orchestra.run.smithi043.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T20:22:56.853 INFO:teuthology.orchestra.run.smithi164.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-04-26T20:22:56.880 INFO:teuthology.orchestra.run.smithi043.stdout:(45/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T20:22:56.921 INFO:teuthology.orchestra.run.smithi043.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 222 kB/s | 135 kB 00:00 2024-04-26T20:22:56.946 INFO:teuthology.orchestra.run.smithi043.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 205 kB/s | 125 kB 00:00 2024-04-26T20:22:56.988 INFO:teuthology.orchestra.run.smithi043.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 215 kB 00:00 2024-04-26T20:22:57.128 INFO:teuthology.orchestra.run.smithi164.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-26T20:22:57.420 INFO:teuthology.orchestra.run.smithi164.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 969 kB/s | 548 kB 00:00 2024-04-26T20:22:57.454 INFO:teuthology.orchestra.run.smithi164.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 962 kB/s | 309 kB 00:00 2024-04-26T20:22:57.487 INFO:teuthology.orchestra.run.smithi164.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 299 kB/s | 19 kB 00:00 2024-04-26T20:22:57.780 INFO:teuthology.orchestra.run.smithi043.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 135 kB/s | 107 kB 00:00 2024-04-26T20:22:57.929 INFO:teuthology.orchestra.run.smithi164.stdout:(107/152): protobuf-compiler-3.14.0-13.el9.x86_ 528 kB/s | 863 kB 00:01 2024-04-26T20:22:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 56 kB/s | 26 kB 00:00 2024-04-26T20:22:58.063 INFO:teuthology.orchestra.run.smithi164.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-04-26T20:22:58.088 INFO:teuthology.orchestra.run.smithi164.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 402 kB/s | 49 kB 00:00 2024-04-26T20:22:58.113 INFO:teuthology.orchestra.run.smithi164.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 462 kB/s | 67 kB 00:00 2024-04-26T20:22:58.147 INFO:teuthology.orchestra.run.smithi164.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-26T20:22:58.239 INFO:teuthology.orchestra.run.smithi164.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 839 kB 00:00 2024-04-26T20:22:58.322 INFO:teuthology.orchestra.run.smithi164.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-26T20:22:58.347 INFO:teuthology.orchestra.run.smithi043.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 38 kB/s | 21 kB 00:00 2024-04-26T20:22:58.356 INFO:teuthology.orchestra.run.smithi164.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 208 kB/s | 43 kB 00:00 2024-04-26T20:22:58.389 INFO:teuthology.orchestra.run.smithi043.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 470 kB/s | 689 kB 00:01 2024-04-26T20:22:58.390 INFO:teuthology.orchestra.run.smithi164.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 213 kB/s | 32 kB 00:00 2024-04-26T20:22:58.415 INFO:teuthology.orchestra.run.smithi164.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 162 kB/s | 15 kB 00:00 2024-04-26T20:22:58.449 INFO:teuthology.orchestra.run.smithi164.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-26T20:22:58.474 INFO:teuthology.orchestra.run.smithi164.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T20:22:58.508 INFO:teuthology.orchestra.run.smithi164.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.4 MB/s | 303 kB 00:00 2024-04-26T20:22:58.525 INFO:teuthology.orchestra.run.smithi164.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 762 kB/s | 35 kB 00:00 2024-04-26T20:22:58.558 INFO:teuthology.orchestra.run.smithi164.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.5 MB/s | 206 kB 00:00 2024-04-26T20:22:58.714 INFO:teuthology.orchestra.run.smithi043.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 534 kB/s | 943 kB 00:01 2024-04-26T20:22:58.808 INFO:teuthology.orchestra.run.smithi164.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-04-26T20:22:58.833 INFO:teuthology.orchestra.run.smithi164.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 472 kB/s | 144 kB 00:00 2024-04-26T20:22:58.850 INFO:teuthology.orchestra.run.smithi164.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 37 kB/s | 11 kB 00:00 2024-04-26T20:22:58.918 INFO:teuthology.orchestra.run.smithi164.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 165 kB/s | 18 kB 00:00 2024-04-26T20:22:58.943 INFO:teuthology.orchestra.run.smithi164.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-26T20:22:58.968 INFO:teuthology.orchestra.run.smithi164.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-26T20:22:58.995 INFO:teuthology.orchestra.run.smithi164.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 258 kB/s | 20 kB 00:00 2024-04-26T20:22:59.039 INFO:teuthology.orchestra.run.smithi043.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-04-26T20:22:59.104 INFO:teuthology.orchestra.run.smithi164.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-26T20:22:59.129 INFO:teuthology.orchestra.run.smithi164.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 290 kB/s | 46 kB 00:00 2024-04-26T20:22:59.154 INFO:teuthology.orchestra.run.smithi164.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 499 kB/s | 79 kB 00:00 2024-04-26T20:22:59.179 INFO:teuthology.orchestra.run.smithi164.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 788 kB/s | 58 kB 00:00 2024-04-26T20:22:59.263 INFO:teuthology.orchestra.run.smithi164.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-26T20:22:59.288 INFO:teuthology.orchestra.run.smithi164.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 124 kB/s | 16 kB 00:00 2024-04-26T20:22:59.313 INFO:teuthology.orchestra.run.smithi164.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 674 kB/s | 90 kB 00:00 2024-04-26T20:22:59.339 INFO:teuthology.orchestra.run.smithi164.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 383 kB/s | 28 kB 00:00 2024-04-26T20:22:59.381 INFO:teuthology.orchestra.run.smithi164.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 470 kB/s | 31 kB 00:00 2024-04-26T20:22:59.398 INFO:teuthology.orchestra.run.smithi043.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 299 kB/s | 313 kB 00:01 2024-04-26T20:22:59.414 INFO:teuthology.orchestra.run.smithi164.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-04-26T20:22:59.439 INFO:teuthology.orchestra.run.smithi164.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 592 kB/s | 59 kB 00:00 2024-04-26T20:22:59.465 INFO:teuthology.orchestra.run.smithi164.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 443 kB/s | 36 kB 00:00 2024-04-26T20:22:59.523 INFO:teuthology.orchestra.run.smithi164.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 621 kB/s | 51 kB 00:00 2024-04-26T20:22:59.557 INFO:teuthology.orchestra.run.smithi164.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-26T20:22:59.582 INFO:teuthology.orchestra.run.smithi164.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 770 kB/s | 90 kB 00:00 2024-04-26T20:22:59.615 INFO:teuthology.orchestra.run.smithi164.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.7 MB/s | 427 kB 00:00 2024-04-26T20:22:59.691 INFO:teuthology.orchestra.run.smithi043.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 28 kB/s | 18 kB 00:00 2024-04-26T20:22:59.708 INFO:teuthology.orchestra.run.smithi164.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 166 kB/s | 20 kB 00:00 2024-04-26T20:22:59.766 INFO:teuthology.orchestra.run.smithi164.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-26T20:22:59.907 INFO:teuthology.orchestra.run.smithi043.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-04-26T20:22:59.933 INFO:teuthology.orchestra.run.smithi164.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-04-26T20:23:00.067 INFO:teuthology.orchestra.run.smithi164.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-04-26T20:23:00.159 INFO:teuthology.orchestra.run.smithi164.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.5 MB/s | 3.4 MB 00:00 2024-04-26T20:23:00.259 INFO:teuthology.orchestra.run.smithi164.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-26T20:23:00.367 INFO:teuthology.orchestra.run.smithi164.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-04-26T20:23:00.378 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:23:00.378 INFO:teuthology.orchestra.run.smithi164.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-26T20:23:00.515 INFO:teuthology.orchestra.run.smithi043.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 194 kB/s | 160 kB 00:00 2024-04-26T20:23:00.590 INFO:teuthology.orchestra.run.smithi043.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-04-26T20:23:01.866 INFO:teuthology.orchestra.run.smithi043.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 457 kB/s | 582 kB 00:01 2024-04-26T20:23:01.933 INFO:teuthology.orchestra.run.smithi043.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 468 kB/s | 662 kB 00:01 2024-04-26T20:23:02.433 INFO:teuthology.orchestra.run.smithi043.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 45 kB/s | 25 kB 00:00 2024-04-26T20:23:02.525 INFO:teuthology.orchestra.run.smithi043.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-04-26T20:23:02.792 INFO:teuthology.orchestra.run.smithi043.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 751 kB/s | 3.0 MB 00:04 2024-04-26T20:23:03.220 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-26T20:23:03.400 INFO:teuthology.orchestra.run.smithi043.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 303 kB/s | 292 kB 00:00 2024-04-26T20:23:03.433 INFO:teuthology.orchestra.run.smithi043.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 208 kB/s | 188 kB 00:00 2024-04-26T20:23:03.458 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-26T20:23:03.459 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-26T20:23:03.459 INFO:teuthology.orchestra.run.smithi043.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-04-26T20:23:05.851 INFO:teuthology.orchestra.run.smithi043.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 680 kB/s | 1.6 MB 00:02 2024-04-26T20:23:06.081 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-26T20:23:06.082 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-26T20:23:07.451 INFO:teuthology.orchestra.run.smithi043.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 570 kB/s | 911 kB 00:01 2024-04-26T20:23:08.185 INFO:teuthology.orchestra.run.smithi043.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 182 kB/s | 133 kB 00:00 2024-04-26T20:23:09.077 INFO:teuthology.orchestra.run.smithi043.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 803 kB/s | 4.4 MB 00:05 2024-04-26T20:23:09.593 INFO:teuthology.orchestra.run.smithi043.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 784 kB/s | 4.7 MB 00:06 2024-04-26T20:23:10.260 INFO:teuthology.orchestra.run.smithi043.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 131 kB/s | 87 kB 00:00 2024-04-26T20:23:10.810 INFO:teuthology.orchestra.run.smithi043.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 597 kB/s | 1.0 MB 00:01 2024-04-26T20:23:11.902 INFO:teuthology.orchestra.run.smithi043.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 224 kB/s | 244 kB 00:01 2024-04-26T20:23:12.421 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-26T20:23:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:23:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:23:12.731 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:23:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:23:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:23:12.980 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:23:13.036 INFO:teuthology.orchestra.run.smithi043.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 219 kB/s | 248 kB 00:01 2024-04-26T20:23:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:23:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:23:13.696 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:23:13.732 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:23:13.836 INFO:teuthology.orchestra.run.smithi043.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 32 kB/s | 26 kB 00:00 2024-04-26T20:23:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:23:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:23:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:23:14.381 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:23:14.411 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:23:14.503 INFO:teuthology.orchestra.run.smithi043.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 29 kB/s | 19 kB 00:00 2024-04-26T20:23:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:23:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:23:14.700 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:23:14.730 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:23:14.804 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:23:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:23:15.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:23:15.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:23:15.445 INFO:teuthology.orchestra.run.smithi043.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 183 kB/s | 172 kB 00:00 2024-04-26T20:23:15.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:23:15.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:23:15.603 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:23:15.688 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:23:15.734 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:23:15.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:23:15.845 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:23:15.878 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:23:15.971 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:23:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:23:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:23:16.125 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:23:16.144 INFO:teuthology.orchestra.run.smithi043.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 50 kB/s | 35 kB 00:00 2024-04-26T20:23:16.249 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:23:16.285 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:23:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:23:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:23:16.473 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:23:16.527 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:23:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:23:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:23:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:23:16.937 INFO:teuthology.orchestra.run.smithi043.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 676 kB/s | 5.8 MB 00:08 2024-04-26T20:23:17.511 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:23:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:23:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:23:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:23:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:23:17.853 INFO:teuthology.orchestra.run.smithi043.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 275 kB/s | 250 kB 00:00 2024-04-26T20:23:18.054 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:23:18.391 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:23:18.461 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:23:18.545 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:23:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:23:18.678 INFO:teuthology.orchestra.run.smithi043.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 270 kB/s | 222 kB 00:00 2024-04-26T20:23:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:23:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:23:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:23:18.839 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:23:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:23:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:23:18.979 INFO:teuthology.orchestra.run.smithi043.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 700 kB/s | 6.0 MB 00:08 2024-04-26T20:23:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:23:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:23:19.240 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:23:19.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:23:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:23:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:23:19.512 INFO:teuthology.orchestra.run.smithi043.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 93 kB/s | 77 kB 00:00 2024-04-26T20:23:19.523 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:23:19.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:23:19.588 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:23:19.696 INFO:teuthology.orchestra.run.smithi043.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 66 kB/s | 42 kB 00:00 2024-04-26T20:23:19.789 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:23:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:23:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:23:20.042 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:23:20.089 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:23:20.153 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:23:20.202 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:23:20.247 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:23:20.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:23:20.369 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:23:20.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:23:20.471 INFO:teuthology.orchestra.run.smithi043.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 281 kB/s | 269 kB 00:00 2024-04-26T20:23:20.513 INFO:teuthology.orchestra.run.smithi043.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 194 kB/s | 159 kB 00:00 2024-04-26T20:23:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:23:20.628 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:23:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:23:20.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:23:20.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:23:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:23:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:23:21.164 INFO:teuthology.orchestra.run.smithi043.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-04-26T20:23:21.472 INFO:teuthology.orchestra.run.smithi043.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 279 kB/s | 279 kB 00:00 2024-04-26T20:23:21.822 INFO:teuthology.orchestra.run.smithi043.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-04-26T20:23:21.959 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:23:22.020 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:23:22.053 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:23:22.175 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:23:22.239 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:23:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:23:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:23:22.448 INFO:teuthology.orchestra.run.smithi043.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-04-26T20:23:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:23:22.635 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:23:22.738 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:23:23.123 INFO:teuthology.orchestra.run.smithi043.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 748 kB/s | 5.1 MB 00:06 2024-04-26T20:23:23.148 INFO:teuthology.orchestra.run.smithi043.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 63 kB/s | 44 kB 00:00 2024-04-26T20:23:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:23:23.723 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:23:23.799 INFO:teuthology.orchestra.run.smithi043.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 55 kB/s | 36 kB 00:00 2024-04-26T20:23:23.966 INFO:teuthology.orchestra.run.smithi043.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 263 kB/s | 221 kB 00:00 2024-04-26T20:23:24.533 INFO:teuthology.orchestra.run.smithi043.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-04-26T20:23:24.975 INFO:teuthology.orchestra.run.smithi043.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 303 kB/s | 305 kB 00:01 2024-04-26T20:23:25.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:23:25.361 INFO:teuthology.orchestra.run.smithi043.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 75 kB/s | 58 kB 00:00 2024-04-26T20:23:25.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:23:25.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:23:25.476 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:23:25.692 INFO:teuthology.orchestra.run.smithi043.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 131 kB/s | 94 kB 00:00 2024-04-26T20:23:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:23:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:23:26.009 INFO:teuthology.orchestra.run.smithi043.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-04-26T20:23:26.050 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:23:26.085 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:23:26.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:23:26.167 INFO:teuthology.orchestra.run.smithi043.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 142 kB/s | 22 kB 00:00 2024-04-26T20:23:26.309 INFO:teuthology.orchestra.run.smithi043.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-04-26T20:23:26.376 INFO:teuthology.orchestra.run.smithi043.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.1 MB/s | 863 kB 00:00 2024-04-26T20:23:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:23:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:23:26.848 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:23:26.875 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:23:27.018 INFO:teuthology.orchestra.run.smithi043.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 482 kB/s | 309 kB 00:00 2024-04-26T20:23:27.052 INFO:teuthology.orchestra.run.smithi043.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 740 kB/s | 548 kB 00:00 2024-04-26T20:23:27.118 INFO:teuthology.orchestra.run.smithi043.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-04-26T20:23:27.219 INFO:teuthology.orchestra.run.smithi043.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-04-26T20:23:27.327 INFO:teuthology.orchestra.run.smithi043.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 452 kB/s | 49 kB 00:00 2024-04-26T20:23:27.494 INFO:teuthology.orchestra.run.smithi043.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T20:23:27.520 INFO:teuthology.orchestra.run.smithi043.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 353 kB/s | 67 kB 00:00 2024-04-26T20:23:27.603 INFO:teuthology.orchestra.run.smithi043.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T20:23:27.720 INFO:teuthology.orchestra.run.smithi043.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-26T20:23:27.770 INFO:teuthology.orchestra.run.smithi043.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-04-26T20:23:27.821 INFO:teuthology.orchestra.run.smithi043.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 437 kB/s | 43 kB 00:00 2024-04-26T20:23:27.871 INFO:teuthology.orchestra.run.smithi043.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-26T20:23:27.944 INFO:teuthology.orchestra.run.smithi043.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 122 kB/s | 15 kB 00:00 2024-04-26T20:23:27.980 INFO:teuthology.orchestra.run.smithi043.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-04-26T20:23:28.055 INFO:teuthology.orchestra.run.smithi043.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T20:23:28.114 INFO:teuthology.orchestra.run.smithi043.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-26T20:23:28.157 INFO:teuthology.orchestra.run.smithi043.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-26T20:23:28.223 INFO:teuthology.orchestra.run.smithi043.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-26T20:23:28.341 INFO:teuthology.orchestra.run.smithi043.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T20:23:28.374 INFO:teuthology.orchestra.run.smithi043.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 965 kB/s | 144 kB 00:00 2024-04-26T20:23:28.441 INFO:teuthology.orchestra.run.smithi043.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T20:23:28.466 INFO:teuthology.orchestra.run.smithi043.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-26T20:23:28.533 INFO:teuthology.orchestra.run.smithi043.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 254 kB/s | 23 kB 00:00 2024-04-26T20:23:28.568 INFO:teuthology.orchestra.run.smithi043.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-26T20:23:28.634 INFO:teuthology.orchestra.run.smithi043.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-26T20:23:28.751 INFO:teuthology.orchestra.run.smithi043.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-26T20:23:28.768 INFO:teuthology.orchestra.run.smithi043.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2024-04-26T20:23:28.868 INFO:teuthology.orchestra.run.smithi043.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 680 kB/s | 79 kB 00:00 2024-04-26T20:23:28.893 INFO:teuthology.orchestra.run.smithi043.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 463 kB/s | 58 kB 00:00 2024-04-26T20:23:28.985 INFO:teuthology.orchestra.run.smithi043.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T20:23:29.011 INFO:teuthology.orchestra.run.smithi043.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-26T20:23:29.111 INFO:teuthology.orchestra.run.smithi043.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 723 kB/s | 90 kB 00:00 2024-04-26T20:23:29.136 INFO:teuthology.orchestra.run.smithi043.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-04-26T20:23:29.212 INFO:teuthology.orchestra.run.smithi043.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-26T20:23:29.245 INFO:teuthology.orchestra.run.smithi043.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T20:23:29.312 INFO:teuthology.orchestra.run.smithi043.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 597 kB/s | 59 kB 00:00 2024-04-26T20:23:29.346 INFO:teuthology.orchestra.run.smithi043.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-04-26T20:23:29.413 INFO:teuthology.orchestra.run.smithi043.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-04-26T20:23:29.455 INFO:teuthology.orchestra.run.smithi043.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T20:23:29.513 INFO:teuthology.orchestra.run.smithi043.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 901 kB/s | 90 kB 00:00 2024-04-26T20:23:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:23:29.580 INFO:teuthology.orchestra.run.smithi043.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-26T20:23:29.604 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:23:29.614 INFO:teuthology.orchestra.run.smithi043.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-04-26T20:23:29.691 INFO:teuthology.orchestra.run.smithi043.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T20:23:29.833 INFO:teuthology.orchestra.run.smithi043.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 7.2 MB/s | 1.6 MB 00:00 2024-04-26T20:23:30.184 INFO:teuthology.orchestra.run.smithi043.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.9 MB/s | 3.4 MB 00:00 2024-04-26T20:23:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:23:30.309 INFO:teuthology.orchestra.run.smithi043.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-04-26T20:23:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:23:30.367 INFO:teuthology.orchestra.run.smithi043.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-26T20:23:30.481 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:23:30.481 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:23:30.481 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:23:30.481 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:30.517 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:23:30.517 INFO:teuthology.orchestra.run.smithi043.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-26T20:23:40.317 INFO:teuthology.orchestra.run.smithi043.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 883 kB/s | 16 MB 00:18 2024-04-26T20:23:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:23:40.330 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.3 MB/s | 222 MB 00:51 2024-04-26T20:23:43.231 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-26T20:23:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:43.494 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-26T20:23:43.495 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-26T20:23:43.538 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:23:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:23:43.585 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:43.585 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:23:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:45.215 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:23:45.295 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:23:45.295 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:45.295 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:23:45.295 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:23:45.295 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:23:45.296 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:45.548 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:23:45.623 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:23:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:23:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:23:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:23:45.624 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:46.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:23:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:23:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:23:46.186 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:23:46.187 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:23:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:46.233 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-26T20:23:46.234 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-26T20:23:46.316 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:23:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:23:46.452 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:23:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:23:46.761 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:23:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:23:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:23:47.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:23:47.187 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:23:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:23:47.259 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:23:47.914 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:23:47.963 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:23:48.021 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:23:48.129 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:23:48.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:23:49.501 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:23:49.541 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:23:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:23:50.683 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:23:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:23:51.007 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:23:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:23:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:51.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:23:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:23:51.356 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:23:51.449 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:23:52.487 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:23:52.530 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:23:52.567 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-26T20:23:52.610 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:23:52.611 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:52.611 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:23:52.611 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:23:52.611 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:23:52.611 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:52.656 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:23:52.682 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:23:52.724 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:23:52.725 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:52.725 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:23:52.725 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:52.771 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:23:52.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:23:52.872 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:23:52.872 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:52.872 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:23:52.873 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:23:52.873 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:23:52.873 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:52.897 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:23:52.956 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:23:53.000 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:23:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:23:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:23:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:23:53.318 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:23:53.318 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:23:53.319 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:23:53.319 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:23:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:23:53.585 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:23:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:23:53.869 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:23:53.898 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:23:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:23:54.081 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:23:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:23:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:23:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:23:54.672 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:23:54.695 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:23:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:23:54.829 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:23:54.899 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:23:55.194 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:23:55.237 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:23:55.425 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:23:55.544 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:23:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:23:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:23:55.751 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:23:55.801 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:23:55.873 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:23:55.922 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:23:55.962 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:23:56.063 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:23:56.133 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:23:56.172 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:23:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:23:56.450 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:23:56.450 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:23:56.450 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:23:56.455 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:23:56.572 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:23:56.636 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:23:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:23:56.912 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:23:56.979 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:23:57.678 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:23:57.789 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:23:57.830 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:23:57.875 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:23:57.924 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:23:57.990 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:23:58.284 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:23:58.341 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:23:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:23:58.421 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:23:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:23:58.516 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:23:58.540 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:23:58.557 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:23:58.586 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:23:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:23:58.635 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:23:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:23:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:23:58.776 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:23:58.843 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:23:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:23:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:23:59.048 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:23:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:23:59.102 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:23:59.140 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:23:59.197 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:23:59.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:23:59.373 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:23:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:23:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:23:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:23:59.685 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:23:59.870 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:23:59.919 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:23:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:23:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:24:00.021 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:00.076 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:00.084 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:24:00.142 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:24:00.221 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:24:00.280 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:24:00.331 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:24:00.407 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:24:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:24:00.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:24:00.666 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:24:00.708 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:24:00.781 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:24:00.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:24:01.005 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:24:01.401 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:24:01.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:24:01.628 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:24:01.812 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:24:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:24:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:24:01.974 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:24:02.008 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:24:02.024 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:24:02.024 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:24:02.062 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:24:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:24:02.164 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:24:02.231 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:24:02.291 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:24:02.294 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:02.400 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:24:02.447 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:24:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:02.547 INFO:teuthology.orchestra.run.smithi164.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:24:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:24:02.718 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:24:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:24:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:24:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:24:05.510 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:24:05.600 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:24:05.640 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:24:05.684 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:24:06.090 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:24:06.129 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:24:06.252 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:24:06.286 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:24:06.347 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:24:06.926 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:24:06.957 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:24:07.058 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:24:07.110 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:24:08.476 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:24:08.476 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:24:08.476 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:24:08.477 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:24:08.478 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:24:08.490 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:24:08.491 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:24:08.492 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:24:08.493 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:24:08.494 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:24:08.495 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:24:08.496 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:24:08.497 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:24:08.498 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:24:08.499 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:24:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:24:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:24:10.580 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:24:10.634 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:24:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:24:10.799 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:24:10.799 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:24:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:10.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:24:10.933 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.935 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.936 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:24:10.937 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:24:10.938 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:24:10.939 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:24:10.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:24:10.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:24:10.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:24:10.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.944 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:24:10.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:10.946 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-26T20:24:11.510 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:24.629 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-04-26T20:24:24.630 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:24.892 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:24:24.942 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:24:24.942 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:24.942 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:24:24.942 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:24:26.708 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:24:27.038 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:24:27.614 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:24:27.614 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:27.614 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:24:27.614 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:24:27.615 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:24:27.615 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:27.753 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:24:27.825 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:24:27.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:24:28.156 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:24:28.295 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:24:28.510 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:24:28.631 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:24:28.818 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:24:28.870 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:24:28.906 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:24:28.938 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:24:29.527 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:24:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:24:29.628 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:24:29.749 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:24:29.784 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:24:31.164 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:24:31.204 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:24:32.371 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:24:32.398 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:24:32.548 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:24:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:24:32.772 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:24:32.852 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:24:32.954 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:24:33.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:24:33.139 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:24:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:24:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:24:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:34.378 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:24:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:24:34.451 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:34.451 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:24:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:24:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:24:34.607 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:34.607 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:24:34.607 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:24:34.608 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:24:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:24:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:40.417 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:24:40.472 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:24:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:24:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:24:40.633 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:24:40.950 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:24:41.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:24:41.042 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:24:41.952 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:41.998 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:24:43.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:24:43.627 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:24:43.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:24:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:24:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:24:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:24:44.073 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:24:44.127 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:24:44.128 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:24:44.268 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:24:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:44.693 INFO:teuthology.orchestra.run.smithi043.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:24:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:24:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:24:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:24:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:24:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:24:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:24:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:24:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:24:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:24:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:24:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:24:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:24:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:24:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:24:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:24:52.683 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:24:52.684 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.685 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:24:52.686 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:24:52.687 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:24:52.688 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:24:52.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:24:52.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:24:52.691 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:24:52.692 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:24:52.693 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.694 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:52.695 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-26T20:24:53.411 DEBUG:teuthology.parallel:result is None 2024-04-26T20:24:53.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:24:53.551 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:24:53.606 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:24:53.606 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:24:53.606 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:24:53.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:24:53.751 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:24:53.822 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:24:53.822 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:24:53.822 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:24:53.824 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T20:24:53.824 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:53.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:24:53.897 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:53.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:24:53.989 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T20:24:53.990 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:53.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:24:54.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:24:54.154 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:54.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:24:54.243 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:24:54.371 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T20:24:54.371 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:54.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:24:54.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:24:54.546 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:54.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:24:54.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:24:54.763 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T20:24:54.763 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:54.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:24:54.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:24:54.946 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:54.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:24:55.034 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:24:55.165 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T20:24:55.245 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T20:24:55.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-04-26T20:24:55.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-04-26T20:24:55.337 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T20:24:55.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:24:55.342 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:24:55.422 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T20:24:55.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:24:55.426 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:24:55.535 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T20:24:55.535 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T20:24:55.535 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T20:24:55.536 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:55.536 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:24:55.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:55.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:55.829 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:55.829 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:24:55.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:56.102 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:24:56.194 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T20:24:56.194 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T20:24:56.195 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': None, 'first_in_suite': False, 'job_id': '7675164', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOu3G4ORMpFFzdyVlekiwhZkIWNMgClxKe2xZr7nLLokggQyFpD3KjXtfiCzRw12iWT0SOiI6I435VB5xx43s8k=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPeG7ialqsj8nIaFb16MqFvM/lqPQZj8cA0W+i2UkjT76rNJuy6/e7d9HlvZuJQyWgpzLSEQ9Ym86BwGF805KHk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T20:24:56.195 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:24:56.200 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:24:56.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:24:56.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:24:56.368 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:56.368 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:24:56.405 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:24:56.406 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-04-26T20:24:56.485 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-26 20:24:45.173961880 +0000 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-26 20:21:53.069331872 +0000 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-26 20:21:53.069331872 +0000 2024-04-26T20:24:56.486 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-26 20:21:53.069331872 +0000 2024-04-26T20:24:56.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:24:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-26T20:24:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-26T20:24:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000246282 s, 2.1 MB/s 2024-04-26T20:24:56.586 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:24:56.631 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-26 20:24:45.174961868 +0000 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-26 20:21:53.678324561 +0000 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-26 20:21:53.678324561 +0000 2024-04-26T20:24:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-26 20:21:53.678324561 +0000 2024-04-26T20:24:56.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:24:56.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-26T20:24:56.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-26T20:24:56.815 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000190018 s, 2.7 MB/s 2024-04-26T20:24:56.819 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:24:56.858 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-04-26T20:24:56.937 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:24:56.937 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:56.937 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-26 20:24:45.175961856 +0000 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-26 20:21:54.288317237 +0000 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-26 20:21:54.288317237 +0000 2024-04-26T20:24:56.938 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-26 20:21:54.288317237 +0000 2024-04-26T20:24:56.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:24:57.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-26T20:24:57.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-26T20:24:57.038 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00017789 s, 2.9 MB/s 2024-04-26T20:24:57.041 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:24:57.083 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-26 20:24:45.175961856 +0000 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-26 20:21:54.926309577 +0000 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-26 20:21:54.926309577 +0000 2024-04-26T20:24:57.161 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-26 20:21:54.926309577 +0000 2024-04-26T20:24:57.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:24:57.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-26T20:24:57.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-26T20:24:57.261 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000201963 s, 2.5 MB/s 2024-04-26T20:24:57.264 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:24:57.305 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:24:57.305 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:57.305 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:24:57.361 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:24:57.362 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-26T20:24:57.413 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 20:24:03.024368822 +0000 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 20:21:47.030670623 +0000 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 20:21:47.030670623 +0000 2024-04-26T20:24:57.414 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 20:21:47.030670623 +0000 2024-04-26T20:24:57.414 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:24:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T20:24:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T20:24:57.489 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000351546 s, 1.5 MB/s 2024-04-26T20:24:57.493 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:24:57.546 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 20:24:03.025368806 +0000 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 20:21:47.621661118 +0000 2024-04-26T20:24:57.595 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 20:21:47.621661118 +0000 2024-04-26T20:24:57.596 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 20:21:47.621661118 +0000 2024-04-26T20:24:57.596 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:24:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T20:24:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T20:24:57.670 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000342336 s, 1.5 MB/s 2024-04-26T20:24:57.674 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:24:57.729 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 20:24:03.026368790 +0000 2024-04-26T20:24:57.776 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 20:21:48.220651484 +0000 2024-04-26T20:24:57.777 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 20:21:48.220651484 +0000 2024-04-26T20:24:57.777 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 20:21:48.220651484 +0000 2024-04-26T20:24:57.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:24:57.846 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T20:24:57.846 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T20:24:57.847 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000319201 s, 1.6 MB/s 2024-04-26T20:24:57.850 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:24:57.907 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-26 20:24:03.027368774 +0000 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-26 20:21:48.821641818 +0000 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-26 20:21:48.821641818 +0000 2024-04-26T20:24:57.954 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-26 20:21:48.821641818 +0000 2024-04-26T20:24:57.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:24:58.072 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-26T20:24:58.073 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-26T20:24:58.073 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.0003632 s, 1.4 MB/s 2024-04-26T20:24:58.076 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:24:58.133 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:24:58.133 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:24:58.133 INFO:tasks.ceph:Generating config... 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] pid file = 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T20:24:58.135 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T20:24:58.135 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T20:24:58.136 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T20:24:58.136 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T20:24:58.136 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T20:24:58.136 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T20:24:58.136 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T20:24:58.137 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T20:24:58.137 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T20:24:58.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T20:24:58.248 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T20:24:58.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T20:24:58.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:24:58.428 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.164'), ('mon.c', '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]')] 2024-04-26T20:24:58.428 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T20:24:58.429 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:58.429 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:24:58.477 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.164 --addv c '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:24:58.607 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:24:58.607 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid 6a354559-7839-4522-9cd7-4f1548bb621f 2024-04-26T20:24:58.607 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-04-26T20:24:58.607 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 6a354559-7839-4522-9cd7-4f1548bb621f 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2024-04-26T20:24:58.607719+0000 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-26T20:24:58.607719+0000 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 17 (quincy) 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] mon.c 2024-04-26T20:24:58.608 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T20:24:58.611 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:24:58.649 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6a354559-7839-4522-9cd7-4f1548bb621f... 2024-04-26T20:24:58.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:24:58.694 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:24:58.805 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-04-26T20:24:58.805 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-04-26T20:24:58.805 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:24:58.805 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:24:58.806 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:24:58.807 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 6a354559-7839-4522-9cd7-4f1548bb621f 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-04-26T20:24:58.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:24:58.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 20 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 20 2024-04-26T20:24:58.810 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 10 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-04-26T20:24:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:24:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon down mkfs grace = 300 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-04-26T20:24:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-04-26T20:24:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-04-26T20:24:58.815 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-04-26T20:24:58.836 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-04-26T20:24:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:24:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.43,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-04-26T20:24:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 6a354559-7839-4522-9cd7-4f1548bb621f 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-04-26T20:24:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-04-26T20:24:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-04-26T20:24:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-04-26T20:24:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:24:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: mon down mkfs grace = 300 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2024-04-26T20:24:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = "" 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2024-04-26T20:24:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-04-26T20:24:58.847 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-04-26T20:24:58.870 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T20:24:58.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T20:24:58.981 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T20:24:58.981 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:58.981 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T20:24:59.024 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:59.024 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T20:24:59.106 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:24:59.106 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:59.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:24:59.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:24:59.234 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:24:59.234 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:24:59.313 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:24:59.313 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:59.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:24:59.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:24:59.431 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:24:59.431 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:24:59.527 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T20:24:59.527 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T20:24:59.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:24:59.650 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:24:59.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:24:59.839 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:24:59.850 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T20:24:59.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:24:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:24:59.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T20:25:00.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:25:00.179 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:25:00.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T20:25:00.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:25:00.417 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:25:00.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T20:25:00.513 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:25:00.656 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:25:00.664 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T20:25:00.748 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T20:25:00.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T20:25:00.864 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T20:25:00.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T20:25:01.010 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T20:25:01.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T20:25:01.173 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T20:25:01.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T20:25:01.324 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T20:25:01.362 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T20:25:01.362 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:25:01.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T20:25:01.456 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:01.457 INFO:tasks.ceph:role: osd.0 2024-04-26T20:25:01.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:25:01.457 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:25:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:01.569 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-26T20:25:01.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-26T20:25:01.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T20:25:01.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T20:25:01.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T20:25:01.817 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:01.817 INFO:tasks.ceph:role: osd.1 2024-04-26T20:25:01.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:25:01.817 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:25:01.891 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:01.892 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:01.925 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-26T20:25:01.931 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-26T20:25:01.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T20:25:02.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T20:25:02.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T20:25:02.182 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:02.182 INFO:tasks.ceph:role: osd.2 2024-04-26T20:25:02.182 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:25:02.182 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:02.263 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:02.264 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:02.264 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:02.264 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:02.296 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-26T20:25:02.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-26T20:25:02.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T20:25:02.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T20:25:02.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T20:25:02.560 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:02.560 INFO:tasks.ceph:role: osd.3 2024-04-26T20:25:02.560 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:25:02.561 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:25:02.629 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:02.630 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:02.663 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-26T20:25:02.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-26T20:25:02.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T20:25:02.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T20:25:02.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:02.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.991+0000 7f6c99616640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T20:25:02.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.992+0000 7f6c99616640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T20:25:02.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.992+0000 7f6c99616640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:25:02.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.992+0000 7f6c99616640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:25:02.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.992+0000 7f6c99616640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:25:02.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:02.996+0000 7f6c99616640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T20:25:06.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T20:25:06.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.241+0000 7fe6b87da640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T20:25:06.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.241+0000 7fe6b87da640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T20:25:06.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.241+0000 7fe6b87da640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:25:06.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.241+0000 7fe6b87da640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:25:06.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.241+0000 7fe6b87da640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:25:06.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:06.243+0000 7fe6b87da640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T20:25:09.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T20:25:09.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:09.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.448+0000 7fbb40e0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T20:25:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.449+0000 7fbb40e0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T20:25:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.449+0000 7fbb40e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:25:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.449+0000 7fbb40e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:25:09.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.449+0000 7fbb40e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:25:09.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:09.451+0000 7fbb40e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T20:25:12.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T20:25:12.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.654+0000 7fc1a3516640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T20:25:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.655+0000 7fc1a3516640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T20:25:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.655+0000 7fc1a3516640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:25:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.655+0000 7fc1a3516640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:25:12.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.655+0000 7fc1a3516640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:25:12.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:12.657+0000 7fc1a3516640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T20:25:15.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T20:25:15.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T20:25:15.821 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:15.821 INFO:tasks.ceph:role: osd.4 2024-04-26T20:25:15.821 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:25:15.822 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:25:15.908 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:15.909 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:16.320 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-26T20:25:16.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-26T20:25:16.326 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T20:25:16.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T20:25:16.543 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T20:25:16.617 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:16.617 INFO:tasks.ceph:role: osd.5 2024-04-26T20:25:16.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:25:16.617 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:16.705 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:16.705 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:16.705 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:17.061 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-26T20:25:17.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-26T20:25:17.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T20:25:17.176 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T20:25:17.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T20:25:17.337 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:17.337 INFO:tasks.ceph:role: osd.6 2024-04-26T20:25:17.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:25:17.337 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:25:17.425 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:17.426 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:17.814 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-26T20:25:17.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-26T20:25:17.820 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T20:25:17.950 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T20:25:18.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T20:25:18.107 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:25:18.107 INFO:tasks.ceph:role: osd.7 2024-04-26T20:25:18.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:25:18.107 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:25:18.194 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:25:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:25:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:25:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:25:18.194 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:25:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:25:18.195 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:25:18.195 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:25:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:25:18.195 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:25:18.603 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-26T20:25:18.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-26T20:25:18.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T20:25:18.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T20:25:18.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:18.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.972+0000 7f0d45a2e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T20:25:18.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.973+0000 7f0d45a2e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T20:25:18.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.973+0000 7f0d45a2e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:25:18.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.973+0000 7f0d45a2e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:25:18.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.973+0000 7f0d45a2e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:25:18.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:18.976+0000 7f0d45a2e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T20:25:21.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T20:25:22.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:22.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.142+0000 7f0a5be16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T20:25:22.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.143+0000 7f0a5be16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T20:25:22.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.143+0000 7f0a5be16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:25:22.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.143+0000 7f0a5be16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:25:22.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.143+0000 7f0a5be16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:25:22.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:22.146+0000 7f0a5be16640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T20:25:25.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T20:25:25.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:25.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.566+0000 7fd1ae3a3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T20:25:25.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.567+0000 7fd1ae3a3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T20:25:25.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.567+0000 7fd1ae3a3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:25:25.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.567+0000 7fd1ae3a3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:25:25.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.567+0000 7fd1ae3a3640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:25:25.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:25.569+0000 7fd1ae3a3640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T20:25:26.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:26.204 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:28.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T20:25:28.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:28.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.757+0000 7f330ec0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T20:25:28.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.758+0000 7f330ec0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T20:25:28.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.758+0000 7f330ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:25:28.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.758+0000 7f330ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:25:28.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.758+0000 7f330ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:25:28.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:28.760+0000 7f330ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T20:25:31.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T20:25:32.051 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T20:25:32.052 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T20:25:32.144 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T20:25:32.236 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T20:25:32.328 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T20:25:32.415 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T20:25:32.505 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T20:25:32.595 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:32.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T20:25:32.683 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:32.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T20:25:32.773 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:32.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T20:25:32.856 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:32.856 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T20:25:32.940 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:32.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T20:25:33.023 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:33.023 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T20:25:33.109 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:33.109 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T20:25:33.191 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:33.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T20:25:33.273 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:33.273 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T20:25:33.333 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:33.333 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T20:25:33.387 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:33.387 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T20:25:33.444 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:33.444 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T20:25:33.499 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T20:25:33.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:25:33.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:25:33.582 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-04-26T20:25:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCbDSxmihfQJhAAxgMUNrLZn63yFfwNrK+4wQ== 2024-04-26T20:25:33.585 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-04-26T20:25:33.585 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCbDSxmhWZpOhAAWb0QM3LzElq9SPNfKOrsog== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCcDSxmiF6yChAAOPapCewm05UD0aJ8Kzfhuw== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCeDSxmteYwOxAA22ydc3JzdGzvj0odfrV7cw== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCiDSxmQ8d2DhAAo+TieWbjX/TLyv9qRUhcaQ== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQClDSxmhs7SGhAAuOsUyG/6tvFoGJz4XlTqEw== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCoDSxmRj4fJxAATdBuaIMmgSayf1Tspziw4w== 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-04-26T20:25:33.586 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCbDSxmuBUJMhAAdVniffQSTS7cMSX18fHwuA== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCcDSxmtuzrGBAA0cLhcnn8XVAs9yI4IKlltQ== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCcDSxmq9ogJxAAueNEX+99za8dVVuMEfLHyA== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCuDSxmrGMJOhAAMxMBzTTKWZ1U9MZUuTgSWQ== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCyDSxmsRiTCBAA4QL+1v1n797ljg+aHww4uw== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQC1DSxmC+PbIRAAaa7nrIFlTNyeXb9PJ/mu6Q== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQC4DSxmbBQ5LRAAtipBsb3oO5d/taQd74X/6w== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCcDSxmFcmKMxAAcYLYOImoPh7zyitYMx2TqQ== 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2024-04-26T20:25:33.587 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCdDSxm8aefABAAhNWzCxvQ5P2w4Qa+4AFTPA== 2024-04-26T20:25:33.588 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-04-26T20:25:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCdDSxmku1jChAAgPKwhMGf35y3TYCNhG76bQ== 2024-04-26T20:25:33.588 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2024-04-26T20:25:33.588 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCdDSxmIQNZExAAlIxAskjGH8Hq83Xxl0cBJw== 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCbDSxmihfQJhAAxgMUNrLZn63yFfwNrK+4wQ== 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCbDSxmhWZpOhAAWb0QM3LzElq9SPNfKOrsog== 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-04-26T20:25:33.589 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCcDSxmiF6yChAAOPapCewm05UD0aJ8Kzfhuw== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCeDSxmteYwOxAA22ydc3JzdGzvj0odfrV7cw== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCiDSxmQ8d2DhAAo+TieWbjX/TLyv9qRUhcaQ== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQClDSxmhs7SGhAAuOsUyG/6tvFoGJz4XlTqEw== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCoDSxmRj4fJxAATdBuaIMmgSayf1Tspziw4w== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCbDSxmuBUJMhAAdVniffQSTS7cMSX18fHwuA== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCcDSxmtuzrGBAA0cLhcnn8XVAs9yI4IKlltQ== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCcDSxmq9ogJxAAueNEX+99za8dVVuMEfLHyA== 2024-04-26T20:25:33.590 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCuDSxmrGMJOhAAMxMBzTTKWZ1U9MZUuTgSWQ== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCyDSxmsRiTCBAA4QL+1v1n797ljg+aHww4uw== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQC1DSxmC+PbIRAAaa7nrIFlTNyeXb9PJ/mu6Q== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQC4DSxmbBQ5LRAAtipBsb3oO5d/taQd74X/6w== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCcDSxmFcmKMxAAcYLYOImoPh7zyitYMx2TqQ== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[client.2] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCdDSxm8aefABAAhNWzCxvQ5P2w4Qa+4AFTPA== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCdDSxmku1jChAAgPKwhMGf35y3TYCNhG76bQ== 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout:[client.3] 2024-04-26T20:25:33.591 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCdDSxmIQNZExAAlIxAskjGH8Hq83Xxl0cBJw== 2024-04-26T20:25:33.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:25:33.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:25:33.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:33.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:33.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:33.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:34.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.207 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.490 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:34.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:25:34.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:25:34.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:34.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:34.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:34.920 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:25:35.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:25:35.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:35.626 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:35.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:35.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:35.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:35.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:36.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:36.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:25:36.192 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T20:25:36.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T20:25:36.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:25:36.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T20:25:36.518 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T20:25:36.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:25:36.758 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T20:25:36.833 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T20:25:36.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:25:37.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T20:25:37.131 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:37.139 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:25:37.190 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T20:25:37.191 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T20:25:37.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T20:25:37.196 INFO:tasks.ceph.mon.a:Started 2024-04-26T20:25:37.196 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T20:25:37.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T20:25:37.201 INFO:tasks.ceph.mon.b:Started 2024-04-26T20:25:37.201 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T20:25:37.201 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T20:25:37.208 INFO:tasks.ceph.mon.c:Started 2024-04-26T20:25:37.208 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T20:25:37.208 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T20:25:37.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T20:25:37.213 INFO:tasks.ceph.mgr.y:Started 2024-04-26T20:25:37.214 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T20:25:37.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T20:25:37.219 INFO:tasks.ceph.mgr.x:Started 2024-04-26T20:25:37.219 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T20:25:37.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T20:25:37.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.394+0000 7f88a7200640 1 Processor -- start 2024-04-26T20:25:37.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.394+0000 7f88a7200640 1 -- start start 2024-04-26T20:25:37.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.395+0000 7f88a7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.395+0000 7f88a7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.395+0000 7f88a7200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a00d1830 con 0x7f88a00d0970 2024-04-26T20:25:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.395+0000 7f88a7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f88a00d0370 con 0x7f88a015d0b0 2024-04-26T20:25:37.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.395+0000 7f88a7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f88a00d1b10 con 0x7f88a00cfc50 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36546/0 (socket says 172.21.15.43:36546) 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 -- 172.21.15.43:0/2808705645 learned_addr learned my addr 172.21.15.43:0/2808705645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 msgr2=0x7f88a00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 msgr2=0x7f88a00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f88a5200640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 msgr2=0x7f88a0070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f88a5200640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 msgr2=0x7f88a0070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:25:37.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.396+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.597+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.597+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.597+0000 7f889fe00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 msgr2=0x7f88a00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f889fe00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 msgr2=0x7f88a00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:25:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:25:37.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f88a5200640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 msgr2=0x7f88a0070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:25:37.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f88a5200640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 msgr2=0x7f88a0070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:25:37.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:25:37.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.598+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:25:37.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.648+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551137073 0 0) 0x7f88a00d1830 con 0x7f88a00d0970 2024-04-26T20:25:37.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.649+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8884003520 con 0x7f88a00d0970 2024-04-26T20:25:37.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2248081699 0 0) 0x7f8884003520 con 0x7f88a00d0970 2024-04-26T20:25:37.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 msgr2=0x7f88a0070060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:25:37.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.164:3300/0 conn(0x7f88a00cfc50 0x7f88a0070060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 msgr2=0x7f88a00ce770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:25:37.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 --2- 172.21.15.43:0/2808705645 >> v2:172.21.15.43:3300/0 conn(0x7f88a015d0b0 0x7f88a00ce770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a00cd9b0 con 0x7f88a00d0970 2024-04-26T20:25:37.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f888c001670 con 0x7f88a00d0970 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.650+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 >> v1:172.21.15.43:6789/0 conn(0x7f88a00d0970 legacy=0x7f88a00cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884003a00 0x7f8884003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 0x7f8884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88840051c0 0x7f888400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00d0370 con 0x7f8884003a00 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00d1b10 con 0x7f8884004800 2024-04-26T20:25:37.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f888400c110 con 0x7f88840051c0 2024-04-26T20:25:37.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.651+0000 7f889fe00640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884003a00 0x7f8884003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.652+0000 7f889f400640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 0x7f8884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.652+0000 7f88a5200640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88840051c0 0x7f888400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f889f400640 1 -- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88840051c0 msgr2=0x7f888400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f889f400640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88840051c0 0x7f888400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f889f400640 1 -- 172.21.15.43:0/2808705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884003a00 msgr2=0x7f8884003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f889f400640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884003a00 0x7f8884003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f889f400640 1 -- 172.21.15.43:0/2808705645 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8884003520 con 0x7f8884004800 2024-04-26T20:25:37.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.653+0000 7f88a5200640 1 -- 172.21.15.43:0/2808705645 reap_dead start 2024-04-26T20:25:37.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.656+0000 7f889f400640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 0x7f8884004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f888c007f70 tx=0x7f8884005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa82d1e63b55bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.657+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c007b90 con 0x7f8884004800 2024-04-26T20:25:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.657+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888c031400 con 0x7f8884004800 2024-04-26T20:25:37.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.657+0000 7f889ea00640 1 -- 172.21.15.43:0/2808705645 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c0317d0 con 0x7f8884004800 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.658+0000 7f88a7200640 1 -- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 msgr2=0x7f8884004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.658+0000 7f88a7200640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 0x7f8884004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f888c007f70 tx=0x7f8884005830 comp rx=0 tx=0).stop 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.658+0000 7f88a7200640 1 -- 172.21.15.43:0/2808705645 shutdown_connections 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.658+0000 7f88a7200640 1 --2- 172.21.15.43:0/2808705645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8884004800 0x7f8884004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.658+0000 7f88a7200640 1 -- 172.21.15.43:0/2808705645 >> 172.21.15.43:0/2808705645 conn(0x7f88a00b48d0 msgr2=0x7f88a00b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.659+0000 7f88a7200640 1 -- 172.21.15.43:0/2808705645 shutdown_connections 2024-04-26T20:25:37.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.659+0000 7f88a7200640 1 -- 172.21.15.43:0/2808705645 wait complete. 2024-04-26T20:25:37.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.662+0000 7f88a7200640 1 Processor -- start 2024-04-26T20:25:37.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.662+0000 7f88a7200640 1 -- start start 2024-04-26T20:25:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88a015c500 0x7f88a014be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88a00c8bd0 0x7f88a014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00ce250 con 0x7f8884004800 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00cdf70 con 0x7f88a015c500 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f88a7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f88a015d490 con 0x7f88a00c8bd0 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f889fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f889fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36578/0 (socket says 172.21.15.43:36578) 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f889fe00640 1 -- 172.21.15.43:0/564270892 learned_addr learned my addr 172.21.15.43:0/564270892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.664+0000 7f889f400640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88a015c500 0x7f88a014be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f88a5200640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88a00c8bd0 0x7f88a014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:37.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 -- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88a00c8bd0 msgr2=0x7f88a014c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88a00c8bd0 0x7f88a014c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 -- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88a015c500 msgr2=0x7f88a014be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88a015c500 0x7f88a014be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 -- 172.21.15.43:0/564270892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a00cc0a0 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.665+0000 7f889fe00640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8890013680 tx=0x7f8890013b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889001a070 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890004230 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000f040 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a00cc380 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a00cc940 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.666+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f889001d070 con 0x7f8884004800 2024-04-26T20:25:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.667+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8890016460 con 0x7f8884004800 2024-04-26T20:25:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.668+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a0069a60 con 0x7f8884004800 2024-04-26T20:25:37.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.671+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8890003610 con 0x7f8884004800 2024-04-26T20:25:37.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.748+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f88a00cf110 con 0x7f8884004800 2024-04-26T20:25:37.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.751+0000 7f8897e00640 1 -- 172.21.15.43:0/564270892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f8890005990 con 0x7f8884004800 2024-04-26T20:25:37.751 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-04-26T20:25:37.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 msgr2=0x7f88a00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8890013680 tx=0x7f8890013b40 comp rx=0 tx=0).stop 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 shutdown_connections 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f88a00c8bd0 0x7f88a014c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88a015c500 0x7f88a014be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 --2- 172.21.15.43:0/564270892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884004800 0x7f88a00cb5e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 >> 172.21.15.43:0/564270892 conn(0x7f88a00b48d0 msgr2=0x7f88a0160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 shutdown_connections 2024-04-26T20:25:37.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:37.753+0000 7f88a7200640 1 -- 172.21.15.43:0/564270892 wait complete. 2024-04-26T20:25:37.772 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T20:25:37.772 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T20:25:37.772 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:37.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T20:25:37.812 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:37.812+0000 7fc305d87040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:25:37.840 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:37.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T20:25:37.958 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:37.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T20:25:38.015 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:38.014+0000 7fc305d87040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:25:38.032 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:25:38.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T20:25:38.085 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:38.084+0000 7fc305d87040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:25:38.104 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:38.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T20:25:38.151 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:38.150+0000 7f381fb24040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:25:38.181 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:38.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T20:25:38.299 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:38.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T20:25:38.370 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:25:38.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T20:25:38.385 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:38.384+0000 7f381fb24040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:25:38.413 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:38.413+0000 7fc305d87040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:25:38.453 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:38.452+0000 7f381fb24040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:25:38.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new cfca3953-6286-4ae6-8653-63d4ec362060 0 2024-04-26T20:25:38.473 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:38.473+0000 7fc305d87040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:25:38.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.677+0000 7fdf78800640 1 Processor -- start 2024-04-26T20:25:38.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.677+0000 7fdf78800640 1 -- start start 2024-04-26T20:25:38.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf78800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7015cca0 0x7fdf700cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf78800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf700d1f40 0x7fdf700d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf78800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf700d12e0 con 0x7fdf70160240 2024-04-26T20:25:38.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf78800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdf700d0240 con 0x7fdf7015cca0 2024-04-26T20:25:38.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf78800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700d0520 con 0x7fdf700d1f40 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf75e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7015cca0 0x7fdf700cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.678+0000 7fdf76800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf700d1f40 0x7fdf700d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf7015cca0 0x7fdf700cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37404/0 (socket says 172.21.15.164:37404) 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 -- 172.21.15.164:0/3276623911 learned_addr learned my addr 172.21.15.164:0/3276623911 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 -- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf700d1f40 msgr2=0x7fdf700d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 --2- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf700d1f40 0x7fdf700d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 -- 172.21.15.164:0/3276623911 >> v1:172.21.15.43:6789/0 conn(0x7fdf70160240 legacy=0x7fdf700d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.679+0000 7fdf75e00640 1 -- 172.21.15.164:0/3276623911 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700d0800 con 0x7fdf7015cca0 2024-04-26T20:25:38.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.680+0000 7fdf75e00640 1 --2- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7015cca0 0x7fdf700cfc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009870 tx=0x7fdf6402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a8e07dcdb4da7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.680+0000 7fdf74a00640 1 -- 172.21.15.164:0/3276623911 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64037070 con 0x7fdf7015cca0 2024-04-26T20:25:38.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.680+0000 7fdf74a00640 1 -- 172.21.15.164:0/3276623911 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf64032440 con 0x7fdf7015cca0 2024-04-26T20:25:38.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.680+0000 7fdf74a00640 1 -- 172.21.15.164:0/3276623911 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64032810 con 0x7fdf7015cca0 2024-04-26T20:25:38.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 -- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7015cca0 msgr2=0x7fdf700cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 --2- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7015cca0 0x7fdf700cfc30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009870 tx=0x7fdf6402a250 comp rx=0 tx=0).stop 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 -- 172.21.15.164:0/3276623911 shutdown_connections 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 --2- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf700d1f40 0x7fdf700d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 --2- 172.21.15.164:0/3276623911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7015cca0 0x7fdf700cfc30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 -- 172.21.15.164:0/3276623911 >> 172.21.15.164:0/3276623911 conn(0x7fdf700d4e40 msgr2=0x7fdf700d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 -- 172.21.15.164:0/3276623911 shutdown_connections 2024-04-26T20:25:38.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.681+0000 7fdf78800640 1 -- 172.21.15.164:0/3276623911 wait complete. 2024-04-26T20:25:38.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.683+0000 7fdf78800640 1 Processor -- start 2024-04-26T20:25:38.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.683+0000 7fdf78800640 1 -- start start 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf700d1f40 0x7fdf701f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 0x7fdf700b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf701606b0 con 0x7fdf7015cca0 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700d0dc0 con 0x7fdf700d1f40 2024-04-26T20:25:38.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf78800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7015d080 con 0x7fdf70160240 2024-04-26T20:25:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf76800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf75e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf700d1f40 0x7fdf701f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf76800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40490/0 (socket says 172.21.15.164:40490) 2024-04-26T20:25:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.684+0000 7fdf76800640 1 -- 172.21.15.164:0/4114237245 learned_addr learned my addr 172.21.15.164:0/4114237245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:38.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf75400640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 0x7fdf700b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 -- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf700d1f40 msgr2=0x7fdf701f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf700d1f40 0x7fdf701f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 -- 172.21.15.164:0/4114237245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 msgr2=0x7fdf700b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 0x7fdf700b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 -- 172.21.15.164:0/4114237245 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700b7330 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf75400640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 0x7fdf700b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf76800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00ef90 tx=0x7fdf6c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c018020 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c014480 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.685+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c009e60 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.686+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf700b6290 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.686+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf700b6850 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.686+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf6c005d10 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.686+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdf6c01dde0 con 0x7fdf70160240 2024-04-26T20:25:38.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.687+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf70069350 con 0x7fdf70160240 2024-04-26T20:25:38.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.690+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdf6c022030 con 0x7fdf70160240 2024-04-26T20:25:38.765 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:38.765+0000 7f381fb24040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:25:38.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.771+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfca3953-6286-4ae6-8653-63d4ec362060", "id": 0} v 0) -- 0x7fdf700c84d0 con 0x7fdf70160240 2024-04-26T20:25:38.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.776+0000 7fdf6a000640 1 -- 172.21.15.164:0/4114237245 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfca3953-6286-4ae6-8653-63d4ec362060", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdf6c010040 con 0x7fdf70160240 2024-04-26T20:25:38.778 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-04-26T20:25:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 msgr2=0x7fdf700c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00ef90 tx=0x7fdf6c00c490 comp rx=0 tx=0).stop 2024-04-26T20:25:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 shutdown_connections 2024-04-26T20:25:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf70160240 0x7fdf700c9e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7015cca0 0x7fdf700b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 --2- 172.21.15.164:0/4114237245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf700d1f40 0x7fdf701f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 >> 172.21.15.164:0/4114237245 conn(0x7fdf700d4e40 msgr2=0x7fdf70155260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 shutdown_connections 2024-04-26T20:25:38.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.778+0000 7fdf78800640 1 -- 172.21.15.164:0/4114237245 wait complete. 2024-04-26T20:25:38.796 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new b828f235-eff0-4dac-9fd9-41ea4af4465f 1 2024-04-26T20:25:38.828 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:38.827+0000 7f381fb24040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:25:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.978+0000 7f086e200640 1 Processor -- start 2024-04-26T20:25:38.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.978+0000 7f086e200640 1 -- start start 2024-04-26T20:25:38.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.979+0000 7f086e200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f086e200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08680cd3b0 con 0x7f08680b43a0 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f086e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08680cd650 con 0x7f08680b2400 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f086e200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08680cdf80 con 0x7f086806e480 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f085fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f08680b2400 0x7f086806dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42200/0 (socket says 172.21.15.164:42200) 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f085fe00640 1 -- 172.21.15.164:0/1815558324 learned_addr learned my addr 172.21.15.164:0/1815558324 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f0867e00640 1 --2- 172.21.15.164:0/1815558324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f0866a00640 1 -- 172.21.15.164:0/1815558324 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497627510 0 0) 0x7f08680cd650 con 0x7f08680b2400 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f0867e00640 1 -- 172.21.15.164:0/1815558324 >> v1:172.21.15.164:6789/0 conn(0x7f08680b2400 legacy=0x7f086806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f0867e00640 1 -- 172.21.15.164:0/1815558324 >> v1:172.21.15.43:6789/0 conn(0x7f08680b43a0 legacy=0x7f08680d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.980+0000 7f0867e00640 1 -- 172.21.15.164:0/1815558324 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680cd930 con 0x7f086806e480 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.981+0000 7f0867e00640 1 --2- 172.21.15.164:0/1815558324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680ce410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860009dc0 tx=0x7f086002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58594dbbbc990d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f0866a00640 1 -- 172.21.15.164:0/1815558324 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860007740 con 0x7f086806e480 2024-04-26T20:25:38.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f0866a00640 1 -- 172.21.15.164:0/1815558324 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08600078e0 con 0x7f086806e480 2024-04-26T20:25:38.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f0866a00640 1 -- 172.21.15.164:0/1815558324 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860007cb0 con 0x7f086806e480 2024-04-26T20:25:38.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 -- 172.21.15.164:0/1815558324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 msgr2=0x7f08680ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 --2- 172.21.15.164:0/1815558324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680ce410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860009dc0 tx=0x7f086002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 -- 172.21.15.164:0/1815558324 shutdown_connections 2024-04-26T20:25:38.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 --2- 172.21.15.164:0/1815558324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680ce410 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 -- 172.21.15.164:0/1815558324 >> 172.21.15.164:0/1815558324 conn(0x7f08680d2d60 msgr2=0x7f08680d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:38.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.982+0000 7f086e200640 1 -- 172.21.15.164:0/1815558324 shutdown_connections 2024-04-26T20:25:38.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.983+0000 7f086e200640 1 -- 172.21.15.164:0/1815558324 wait complete. 2024-04-26T20:25:38.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.984+0000 7f086e200640 1 Processor -- start 2024-04-26T20:25:38.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.984+0000 7f086e200640 1 -- start start 2024-04-26T20:25:38.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 0x7f086815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08680b43a0 0x7f08680b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:38.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f08680d3aa0 con 0x7f08680b2400 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08680d21c0 con 0x7f08680b43a0 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f086e200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08680ce950 con 0x7f086806e480 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f0867400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f0867e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08680b43a0 0x7f08680b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f0867400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40498/0 (socket says 172.21.15.164:40498) 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.985+0000 7f0867400640 1 -- 172.21.15.164:0/1795860255 learned_addr learned my addr 172.21.15.164:0/1795860255 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f085fe00640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 0x7f086815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 -- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08680b43a0 msgr2=0x7f08680b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08680b43a0 0x7f08680b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 -- 172.21.15.164:0/1795860255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 msgr2=0x7f086815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 0x7f086815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 -- 172.21.15.164:0/1795860255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680b6bc0 con 0x7f086806e480 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f085fe00640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 0x7f086815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f0867400640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f085800b420 tx=0x7f085800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:38.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858011830 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858016070 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858011c10 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.986+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08680b6ea0 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.987+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f086815fbf0 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.987+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0858004020 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.987+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f085801b460 con 0x7f086806e480 2024-04-26T20:25:38.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.988+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868069350 con 0x7f086806e480 2024-04-26T20:25:38.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:38.990+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0858020020 con 0x7f086806e480 2024-04-26T20:25:39.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.070+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b828f235-eff0-4dac-9fd9-41ea4af4465f", "id": 1} v 0) -- 0x7f08680b38a0 con 0x7f086806e480 2024-04-26T20:25:39.076 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.076+0000 7f085f400640 1 -- 172.21.15.164:0/1795860255 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b828f235-eff0-4dac-9fd9-41ea4af4465f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0858011db0 con 0x7f086806e480 2024-04-26T20:25:39.077 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 msgr2=0x7f08680c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f085800b420 tx=0x7f085800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 shutdown_connections 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08680b43a0 0x7f08680b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08680b2400 0x7f086815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 --2- 172.21.15.164:0/1795860255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f086806e480 0x7f08680c96b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 >> 172.21.15.164:0/1795860255 conn(0x7f08680d2d60 msgr2=0x7f08680cf5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 shutdown_connections 2024-04-26T20:25:39.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.077+0000 7f086e200640 1 -- 172.21.15.164:0/1795860255 wait complete. 2024-04-26T20:25:39.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 1de917eb-de32-46d1-aa44-506a7a1619a5 2 2024-04-26T20:25:39.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.316+0000 7f12e9600640 1 Processor -- start 2024-04-26T20:25:39.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.316+0000 7f12e9600640 1 -- start start 2024-04-26T20:25:39.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12e9600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 0x7f12e416a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12e9600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e40d12e0 con 0x7f12e415cca0 2024-04-26T20:25:39.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12e9600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e40d0240 con 0x7f12e40cfc30 2024-04-26T20:25:39.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12e9600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40d0520 con 0x7f12e40d1f40 2024-04-26T20:25:39.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12dbe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f12e40cfc30 0x7f12e40d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42208/0 (socket says 172.21.15.164:42208) 2024-04-26T20:25:39.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12dbe00640 1 -- 172.21.15.164:0/37411275 learned_addr learned my addr 172.21.15.164:0/37411275 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-26T20:25:39.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.317+0000 7f12e3400640 1 --2- 172.21.15.164:0/37411275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 0x7f12e416a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.318+0000 7f12e3400640 1 -- 172.21.15.164:0/37411275 >> v1:172.21.15.164:6789/0 conn(0x7f12e40cfc30 legacy=0x7f12e40d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.318+0000 7f12e3400640 1 -- 172.21.15.164:0/37411275 >> v1:172.21.15.43:6789/0 conn(0x7f12e415cca0 legacy=0x7f12e4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.318+0000 7f12e3400640 1 -- 172.21.15.164:0/37411275 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e40d0800 con 0x7f12e40d1f40 2024-04-26T20:25:39.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.319+0000 7f12e3400640 1 --2- 172.21.15.164:0/37411275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 0x7f12e416a140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009dc0 tx=0x7f12dc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbec40e6d7845970 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.319+0000 7f12e2000640 1 -- 172.21.15.164:0/37411275 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc007740 con 0x7f12e40d1f40 2024-04-26T20:25:39.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.319+0000 7f12e2000640 1 -- 172.21.15.164:0/37411275 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc0078e0 con 0x7f12e40d1f40 2024-04-26T20:25:39.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.319+0000 7f12e2000640 1 -- 172.21.15.164:0/37411275 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc007cb0 con 0x7f12e40d1f40 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 -- 172.21.15.164:0/37411275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 msgr2=0x7f12e416a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 --2- 172.21.15.164:0/37411275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 0x7f12e416a140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009dc0 tx=0x7f12dc02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 -- 172.21.15.164:0/37411275 shutdown_connections 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 --2- 172.21.15.164:0/37411275 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e40d1f40 0x7f12e416a140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 -- 172.21.15.164:0/37411275 >> 172.21.15.164:0/37411275 conn(0x7f12e40d4e40 msgr2=0x7f12e40d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 -- 172.21.15.164:0/37411275 shutdown_connections 2024-04-26T20:25:39.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.320+0000 7f12e9600640 1 -- 172.21.15.164:0/37411275 wait complete. 2024-04-26T20:25:39.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.323+0000 7f12e9600640 1 Processor -- start 2024-04-26T20:25:39.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.323+0000 7f12e9600640 1 -- start start 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12e40cfc30 0x7f12e40b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 0x7f12e40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40d3a20 con 0x7f12e40cfc30 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40d0dc0 con 0x7f12e40d1f40 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e9600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40d3ba0 con 0x7f12e415cca0 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e3400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 0x7f12e40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37440/0 (socket says 172.21.15.164:37440) 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12dbe00640 1 -- 172.21.15.164:0/3601830024 learned_addr learned my addr 172.21.15.164:0/3601830024 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.324+0000 7f12e2a00640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12e40cfc30 0x7f12e40b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.326 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 -- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 msgr2=0x7f12e4144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 -- 172.21.15.164:0/3601830024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12e40cfc30 msgr2=0x7f12e40b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12e40cfc30 0x7f12e40b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 -- 172.21.15.164:0/3601830024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4145360 con 0x7f12e415cca0 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12dbe00640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:39.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e3400640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 0x7f12e40ca6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12e40d4800 tx=0x7f12dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc040070 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc031bc0 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc045460 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.325+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e40cb860 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.326+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e40cb2a0 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.326+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12dc039040 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.326+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f12dc045a20 con 0x7f12e415cca0 2024-04-26T20:25:39.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.326+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e4069350 con 0x7f12e415cca0 2024-04-26T20:25:39.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.329+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f12dc045600 con 0x7f12e415cca0 2024-04-26T20:25:39.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.406+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1de917eb-de32-46d1-aa44-506a7a1619a5", "id": 2} v 0) -- 0x7f12e4074790 con 0x7f12e415cca0 2024-04-26T20:25:39.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.412+0000 7f12daa00640 1 -- 172.21.15.164:0/3601830024 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1de917eb-de32-46d1-aa44-506a7a1619a5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f12dc058480 con 0x7f12e415cca0 2024-04-26T20:25:39.413 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.413+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 msgr2=0x7f12e40ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.413+0000 7f12e9600640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 0x7f12e40ca6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12e40d4800 tx=0x7f12dc004a40 comp rx=0 tx=0).stop 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 shutdown_connections 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12e415cca0 0x7f12e40ca6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12e40d1f40 0x7f12e4144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 --2- 172.21.15.164:0/3601830024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12e40cfc30 0x7f12e40b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 >> 172.21.15.164:0/3601830024 conn(0x7f12e40d4e40 msgr2=0x7f12e415d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 shutdown_connections 2024-04-26T20:25:39.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.414+0000 7f12e9600640 1 -- 172.21.15.164:0/3601830024 wait complete. 2024-04-26T20:25:39.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new f9fc22c8-e0be-4eaf-8351-157dcfd186e0 3 2024-04-26T20:25:39.476 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.475+0000 7fc305d87040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:25:39.584 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.584+0000 7fc305d87040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:25:39.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.610+0000 7f5342000640 1 Processor -- start 2024-04-26T20:25:39.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.610+0000 7f5342000640 1 -- start start 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f5342000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f533c072e30 0x7f533c073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f5342000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f533c13f410 0x7f533c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f5342000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533c073750 con 0x7f533c1410f0 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f5342000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f533c13f160 con 0x7f533c072e30 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f5342000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f533c1415b0 con 0x7f533c13f410 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f533aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f533c13f410 0x7f533c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f533aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f533c13f410 0x7f533c0b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37454/0 (socket says 172.21.15.164:37454) 2024-04-26T20:25:39.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f533aa00640 1 -- 172.21.15.164:0/112976755 learned_addr learned my addr 172.21.15.164:0/112976755 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:39.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.612+0000 7f533b400640 1 --2- 172.21.15.164:0/112976755 >> v2:172.21.15.43:3300/0 conn(0x7f533c072e30 0x7f533c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.613+0000 7f533aa00640 1 -- 172.21.15.164:0/112976755 >> v2:172.21.15.43:3300/0 conn(0x7f533c072e30 msgr2=0x7f533c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:39.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.613+0000 7f533aa00640 1 --2- 172.21.15.164:0/112976755 >> v2:172.21.15.43:3300/0 conn(0x7f533c072e30 0x7f533c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.613+0000 7f533aa00640 1 -- 172.21.15.164:0/112976755 >> v1:172.21.15.43:6789/0 conn(0x7f533c1410f0 legacy=0x7f533c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.613+0000 7f533aa00640 1 -- 172.21.15.164:0/112976755 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c141890 con 0x7f533c13f410 2024-04-26T20:25:39.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.614+0000 7f533aa00640 1 --2- 172.21.15.164:0/112976755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c13f410 0x7f533c0b62a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f533c072bf0 tx=0x7f532802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac985b65daf7492 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.615+0000 7f533a000640 1 -- 172.21.15.164:0/112976755 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328037070 con 0x7f533c13f410 2024-04-26T20:25:39.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.615+0000 7f533a000640 1 -- 172.21.15.164:0/112976755 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5328032440 con 0x7f533c13f410 2024-04-26T20:25:39.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.615+0000 7f533a000640 1 -- 172.21.15.164:0/112976755 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328032810 con 0x7f533c13f410 2024-04-26T20:25:39.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.615+0000 7f5342000640 1 -- 172.21.15.164:0/112976755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c13f410 msgr2=0x7f533c0b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.615+0000 7f5342000640 1 --2- 172.21.15.164:0/112976755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c13f410 0x7f533c0b62a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f533c072bf0 tx=0x7f532802a270 comp rx=0 tx=0).stop 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 -- 172.21.15.164:0/112976755 shutdown_connections 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 --2- 172.21.15.164:0/112976755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c13f410 0x7f533c0b62a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 --2- 172.21.15.164:0/112976755 >> v2:172.21.15.43:3300/0 conn(0x7f533c072e30 0x7f533c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 -- 172.21.15.164:0/112976755 >> 172.21.15.164:0/112976755 conn(0x7f533c0b38a0 msgr2=0x7f533c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 -- 172.21.15.164:0/112976755 shutdown_connections 2024-04-26T20:25:39.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.616+0000 7f5342000640 1 -- 172.21.15.164:0/112976755 wait complete. 2024-04-26T20:25:39.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.617+0000 7f5342000640 1 Processor -- start 2024-04-26T20:25:39.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.617+0000 7f5342000640 1 -- start start 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 0x7f533c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 0x7f533c1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f533c142130 con 0x7f533c072e30 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f533c141e50 con 0x7f533c1410f0 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f5342000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f533c0b5f10 con 0x7f533c13f410 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.618+0000 7f533be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37466/0 (socket says 172.21.15.164:37466) 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 -- 172.21.15.164:0/3354104098 learned_addr learned my addr 172.21.15.164:0/3354104098 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533aa00640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 0x7f533c1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533b400640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 0x7f533c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 -- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 msgr2=0x7f533c1506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 0x7f533c1506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 -- 172.21.15.164:0/3354104098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 msgr2=0x7f533c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 0x7f533c151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 -- 172.21.15.164:0/3354104098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c136060 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533b400640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 0x7f533c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533aa00640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 0x7f533c1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.619+0000 7f533be00640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f533400c730 tx=0x7f533400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334003b70 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5334003d30 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334005310 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c136340 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c136900 con 0x7f533c1410f0 2024-04-26T20:25:39.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53340054b0 con 0x7f533c1410f0 2024-04-26T20:25:39.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.620+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f53340057a0 con 0x7f533c1410f0 2024-04-26T20:25:39.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.621+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533c069a60 con 0x7f533c1410f0 2024-04-26T20:25:39.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.624+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f533400a640 con 0x7f533c1410f0 2024-04-26T20:25:39.637 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.637+0000 7fc305d87040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:25:39.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.704+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f9fc22c8-e0be-4eaf-8351-157dcfd186e0", "id": 3} v 0) -- 0x7f533c138ec0 con 0x7f533c1410f0 2024-04-26T20:25:39.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.710+0000 7f5333400640 1 -- 172.21.15.164:0/3354104098 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9fc22c8-e0be-4eaf-8351-157dcfd186e0", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f533400a860 con 0x7f533c1410f0 2024-04-26T20:25:39.711 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-04-26T20:25:39.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 msgr2=0x7f533c15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f533400c730 tx=0x7f533400cbf0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 shutdown_connections 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f533c13f410 0x7f533c1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f533c1410f0 0x7f533c15c380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 --2- 172.21.15.164:0/3354104098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f533c072e30 0x7f533c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.711+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 >> 172.21.15.164:0/3354104098 conn(0x7f533c0b38a0 msgr2=0x7f533c0b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.712+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 shutdown_connections 2024-04-26T20:25:39.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.712+0000 7f5342000640 1 -- 172.21.15.164:0/3354104098 wait complete. 2024-04-26T20:25:39.729 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 8d25a30b-4f74-4152-a9d9-765b60a96f2b 4 2024-04-26T20:25:39.773 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:39.772+0000 7f381fb24040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:25:39.838 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.837+0000 7fc305d87040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:25:39.881 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:39.880+0000 7f381fb24040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:25:39.901 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.900+0000 7fc305d87040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:25:39.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.906+0000 7faa54a00640 1 Processor -- start 2024-04-26T20:25:39.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.907+0000 7faa54a00640 1 -- start start 2024-04-26T20:25:39.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa54a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 0x7faa500d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa54a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 0x7faa5006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa54a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa50070ad0 con 0x7faa500d22f0 2024-04-26T20:25:39.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa54a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7faa500ce7f0 con 0x7faa5006c350 2024-04-26T20:25:39.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa54a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa500ce230 con 0x7faa500704c0 2024-04-26T20:25:39.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.908+0000 7faa47e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 0x7faa500d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa4f400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7faa500d22f0 0x7faa5015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42220/0 (socket says 172.21.15.164:42220) 2024-04-26T20:25:39.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa4f400640 1 -- 172.21.15.164:0/1534838263 learned_addr learned my addr 172.21.15.164:0/1534838263 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-26T20:25:39.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa4fe00640 1 --2- 172.21.15.164:0/1534838263 >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 0x7faa5006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa47e00640 1 -- 172.21.15.164:0/1534838263 >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 msgr2=0x7faa5006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa47e00640 1 --2- 172.21.15.164:0/1534838263 >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 0x7faa5006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa47e00640 1 -- 172.21.15.164:0/1534838263 >> v1:172.21.15.164:6789/0 conn(0x7faa500d22f0 legacy=0x7faa5015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa47e00640 1 -- 172.21.15.164:0/1534838263 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa500ce510 con 0x7faa500704c0 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa4ea00640 1 -- 172.21.15.164:0/1534838263 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213129148 0 0) 0x7faa50070ad0 con 0x7faa500d22f0 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.909+0000 7faa4fe00640 1 --2- 172.21.15.164:0/1534838263 >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 0x7faa5006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:39.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.910+0000 7faa47e00640 1 --2- 172.21.15.164:0/1534838263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 0x7faa500d1540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa380097a0 tx=0x7faa3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4320bad5a7128dab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.910+0000 7faa4ea00640 1 -- 172.21.15.164:0/1534838263 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38037070 con 0x7faa500704c0 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.910+0000 7faa4ea00640 1 -- 172.21.15.164:0/1534838263 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa38032440 con 0x7faa500704c0 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.910+0000 7faa4ea00640 1 -- 172.21.15.164:0/1534838263 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38032810 con 0x7faa500704c0 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 -- 172.21.15.164:0/1534838263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 msgr2=0x7faa500d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 --2- 172.21.15.164:0/1534838263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 0x7faa500d1540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa380097a0 tx=0x7faa3802a180 comp rx=0 tx=0).stop 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 -- 172.21.15.164:0/1534838263 shutdown_connections 2024-04-26T20:25:39.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 --2- 172.21.15.164:0/1534838263 >> v2:172.21.15.43:3300/0 conn(0x7faa5006c350 0x7faa5006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 --2- 172.21.15.164:0/1534838263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500704c0 0x7faa500d1540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 -- 172.21.15.164:0/1534838263 >> 172.21.15.164:0/1534838263 conn(0x7faa50068090 msgr2=0x7faa50068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 -- 172.21.15.164:0/1534838263 shutdown_connections 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.911+0000 7faa54a00640 1 -- 172.21.15.164:0/1534838263 wait complete. 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.913+0000 7faa54a00640 1 Processor -- start 2024-04-26T20:25:39.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.913+0000 7faa54a00640 1 -- start start 2024-04-26T20:25:39.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa500704c0 0x7faa5015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 0x7faa5014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa500d2850 con 0x7faa500704c0 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa500d26d0 con 0x7faa5006c350 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa54a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa500d0420 con 0x7faa500d22f0 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa4f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa4f400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37468/0 (socket says 172.21.15.164:37468) 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa4f400640 1 -- 172.21.15.164:0/3694035881 learned_addr learned my addr 172.21.15.164:0/3694035881 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa4fe00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 0x7faa5014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.914+0000 7faa47e00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa500704c0 0x7faa5015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 -- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 msgr2=0x7faa500b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 -- 172.21.15.164:0/3694035881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa500704c0 msgr2=0x7faa5015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:39.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa500704c0 0x7faa5015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:39.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 -- 172.21.15.164:0/3694035881 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa501f4ce0 con 0x7faa500d22f0 2024-04-26T20:25:39.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4f400640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:39.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa4fe00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 0x7faa5014fe50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa4800ce20 tx=0x7faa4800a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4801a070 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4800d840 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa500b5310 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800dc30 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa500b58d0 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.915+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa4801d070 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.916+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7faa4800ddd0 con 0x7faa500d22f0 2024-04-26T20:25:39.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.916+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa500506c0 con 0x7faa500d22f0 2024-04-26T20:25:39.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.919+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7faa48024070 con 0x7faa500d22f0 2024-04-26T20:25:39.935 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:39.935+0000 7f381fb24040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:25:39.964 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:39.964+0000 7fc305d87040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:25:39.999 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:39.998+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8d25a30b-4f74-4152-a9d9-765b60a96f2b", "id": 4} v 0) -- 0x7faa5014fc10 con 0x7faa500d22f0 2024-04-26T20:25:40.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.004+0000 7faa47400640 1 -- 172.21.15.164:0/3694035881 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d25a30b-4f74-4152-a9d9-765b60a96f2b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7faa4801f440 con 0x7faa500d22f0 2024-04-26T20:25:40.005 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-04-26T20:25:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 msgr2=0x7faa5014fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 0x7faa5014fe50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa4800ce20 tx=0x7faa4800a710 comp rx=0 tx=0).stop 2024-04-26T20:25:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 shutdown_connections 2024-04-26T20:25:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa500d22f0 0x7faa5014fe50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa500704c0 0x7faa5015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 --2- 172.21.15.164:0/3694035881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa5006c350 0x7faa500b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 >> 172.21.15.164:0/3694035881 conn(0x7faa50068090 msgr2=0x7faa5006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 shutdown_connections 2024-04-26T20:25:40.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.005+0000 7faa54a00640 1 -- 172.21.15.164:0/3694035881 wait complete. 2024-04-26T20:25:40.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new a14b82be-18ac-4729-b317-820a65803caa 5 2024-04-26T20:25:40.098 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:40.098+0000 7fc305d87040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:25:40.136 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:40.135+0000 7f381fb24040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:25:40.202 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:40.201+0000 7f381fb24040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:25:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.242+0000 7f2e08e00640 1 Processor -- start 2024-04-26T20:25:40.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.242+0000 7f2e08e00640 1 -- start start 2024-04-26T20:25:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e08e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 0x7f2e040d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e08e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e08e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e040d2d70 con 0x7f2e040b3cb0 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e08e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0415f560 con 0x7f2e0415f180 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e08e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2e040d06e0 con 0x7f2e040d3830 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40540/0 (socket says 172.21.15.164:40540) 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.244+0000 7f2e03e00640 1 -- 172.21.15.164:0/626859507 learned_addr learned my addr 172.21.15.164:0/626859507 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03400640 1 --2- 172.21.15.164:0/626859507 >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 0x7f2e040d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03e00640 1 -- 172.21.15.164:0/626859507 >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 msgr2=0x7f2e040d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03e00640 1 --2- 172.21.15.164:0/626859507 >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 0x7f2e040d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03e00640 1 -- 172.21.15.164:0/626859507 >> v1:172.21.15.43:6789/0 conn(0x7f2e040b3cb0 legacy=0x7f2e040d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03e00640 1 -- 172.21.15.164:0/626859507 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e040d09c0 con 0x7f2e040d3830 2024-04-26T20:25:40.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.245+0000 7f2e03400640 1 --2- 172.21.15.164:0/626859507 >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 0x7f2e040d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:40.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.246+0000 7f2e03e00640 1 --2- 172.21.15.164:0/626859507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216d93873c2b5f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.246+0000 7f2e02000640 1 -- 172.21.15.164:0/626859507 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc036070 con 0x7f2e040d3830 2024-04-26T20:25:40.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.246+0000 7f2e02000640 1 -- 172.21.15.164:0/626859507 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc031600 con 0x7f2e040d3830 2024-04-26T20:25:40.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.246+0000 7f2e02000640 1 -- 172.21.15.164:0/626859507 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0319d0 con 0x7f2e040d3830 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 -- 172.21.15.164:0/626859507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 msgr2=0x7f2e040b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 --2- 172.21.15.164:0/626859507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 -- 172.21.15.164:0/626859507 shutdown_connections 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 --2- 172.21.15.164:0/626859507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e040d3830 0x7f2e040b1740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 --2- 172.21.15.164:0/626859507 >> v2:172.21.15.43:3300/0 conn(0x7f2e0415f180 0x7f2e040d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 -- 172.21.15.164:0/626859507 >> 172.21.15.164:0/626859507 conn(0x7f2e040d4bd0 msgr2=0x7f2e040d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 -- 172.21.15.164:0/626859507 shutdown_connections 2024-04-26T20:25:40.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.247+0000 7f2e08e00640 1 -- 172.21.15.164:0/626859507 wait complete. 2024-04-26T20:25:40.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.248+0000 7f2e08e00640 1 Processor -- start 2024-04-26T20:25:40.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.249+0000 7f2e08e00640 1 -- start start 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.249+0000 7f2e08e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 0x7f2e04159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.249+0000 7f2e08e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 0x7f2e040b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e08e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e08e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2e040d11c0 con 0x7f2e040b3cb0 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e08e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2e040d2a40 con 0x7f2e040d3830 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e08e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2e041570f0 con 0x7f2e0415f180 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e02a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 0x7f2e040b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40556/0 (socket says 172.21.15.164:40556) 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 -- 172.21.15.164:0/2152099379 learned_addr learned my addr 172.21.15.164:0/2152099379 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03400640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 0x7f2e04159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 -- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 msgr2=0x7f2e040b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 0x7f2e040b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 -- 172.21.15.164:0/2152099379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 msgr2=0x7f2e04159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 0x7f2e04159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03e00640 1 -- 172.21.15.164:0/2152099379 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e041573d0 con 0x7f2e0415f180 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e02a00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 0x7f2e040b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:40.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.250+0000 7f2e03400640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 0x7f2e04159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:40.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2e03e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc034040 tx=0x7f2dfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc036050 con 0x7f2e0415f180 2024-04-26T20:25:40.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc042070 con 0x7f2e0415f180 2024-04-26T20:25:40.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e041576b0 con 0x7f2e0415f180 2024-04-26T20:25:40.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc009510 con 0x7f2e0415f180 2024-04-26T20:25:40.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e040b46f0 con 0x7f2e0415f180 2024-04-26T20:25:40.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2dfc007bf0 con 0x7f2e0415f180 2024-04-26T20:25:40.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.251+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2dfc047be0 con 0x7f2e0415f180 2024-04-26T20:25:40.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.252+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e040506c0 con 0x7f2e0415f180 2024-04-26T20:25:40.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.254+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2dfc047460 con 0x7f2e0415f180 2024-04-26T20:25:40.268 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:40.267+0000 7f381fb24040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:25:40.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.333+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a14b82be-18ac-4729-b317-820a65803caa", "id": 5} v 0) -- 0x7f2e040d3480 con 0x7f2e0415f180 2024-04-26T20:25:40.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.342+0000 7f2dfb400640 1 -- 172.21.15.164:0/2152099379 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a14b82be-18ac-4729-b317-820a65803caa", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2dfc047640 con 0x7f2e0415f180 2024-04-26T20:25:40.343 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-04-26T20:25:40.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.343+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 msgr2=0x7f2e04158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.343+0000 7f2e08e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc034040 tx=0x7f2dfc004a40 comp rx=0 tx=0).stop 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 shutdown_connections 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e0415f180 0x7f2e04158190 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e040d3830 0x7f2e040b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 --2- 172.21.15.164:0/2152099379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e040b3cb0 0x7f2e04159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 >> 172.21.15.164:0/2152099379 conn(0x7f2e040d4bd0 msgr2=0x7f2e040d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 shutdown_connections 2024-04-26T20:25:40.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.344+0000 7f2e08e00640 1 -- 172.21.15.164:0/2152099379 wait complete. 2024-04-26T20:25:40.359 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new decfbdf4-6d09-495f-81a4-0935e7bae1b1 6 2024-04-26T20:25:40.403 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:40.402+0000 7f381fb24040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:25:40.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.533+0000 7fbab6000640 1 Processor -- start 2024-04-26T20:25:40.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.534+0000 7fbab6000640 1 -- start start 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbab6000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbab015ef30 0x7fbab00d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbab6000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbab015f310 0x7fbab00d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbab6000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab00d3a70 con 0x7fbab00d0600 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbab6000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbab00d3840 con 0x7fbab015ef30 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbab6000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbab00cfb20 con 0x7fbab015f310 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbaafe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbab015f310 0x7fbab00d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.535+0000 7fbaafe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbab015f310 0x7fbab00d3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37480/0 (socket says 172.21.15.164:37480) 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbaafe00640 1 -- 172.21.15.164:0/184486666 learned_addr learned my addr 172.21.15.164:0/184486666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbab5600640 1 --1- 172.21.15.164:0/184486666 >> v1:172.21.15.164:6789/0 conn(0x7fbab00d0600 0x7fbab0162c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42236/0 (socket says 172.21.15.164:42236) 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbab4c00640 1 --2- 172.21.15.164:0/184486666 >> v2:172.21.15.43:3300/0 conn(0x7fbab015ef30 0x7fbab00d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbaafe00640 1 -- 172.21.15.164:0/184486666 >> v2:172.21.15.43:3300/0 conn(0x7fbab015ef30 msgr2=0x7fbab00d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:40.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbaafe00640 1 --2- 172.21.15.164:0/184486666 >> v2:172.21.15.43:3300/0 conn(0x7fbab015ef30 0x7fbab00d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbaafe00640 1 -- 172.21.15.164:0/184486666 >> v1:172.21.15.164:6789/0 conn(0x7fbab00d0600 legacy=0x7fbab0162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.536+0000 7fbaafe00640 1 -- 172.21.15.164:0/184486666 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab00cfe00 con 0x7fbab015f310 2024-04-26T20:25:40.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.537+0000 7fbaafe00640 1 --2- 172.21.15.164:0/184486666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015f310 0x7fbab00d3300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbab00d2820 tx=0x7fba9c02a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fc1921cfbec5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.538+0000 7fbaaf400640 1 -- 172.21.15.164:0/184486666 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c037070 con 0x7fbab015f310 2024-04-26T20:25:40.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.538+0000 7fbaaf400640 1 -- 172.21.15.164:0/184486666 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c032440 con 0x7fbab015f310 2024-04-26T20:25:40.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.538+0000 7fbaaf400640 1 -- 172.21.15.164:0/184486666 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c032810 con 0x7fbab015f310 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.538+0000 7fbab6000640 1 -- 172.21.15.164:0/184486666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015f310 msgr2=0x7fbab00d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 --2- 172.21.15.164:0/184486666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015f310 0x7fbab00d3300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbab00d2820 tx=0x7fba9c02a200 comp rx=0 tx=0).stop 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 -- 172.21.15.164:0/184486666 shutdown_connections 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 --2- 172.21.15.164:0/184486666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015f310 0x7fbab00d3300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 --2- 172.21.15.164:0/184486666 >> v2:172.21.15.43:3300/0 conn(0x7fbab015ef30 0x7fbab00d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 -- 172.21.15.164:0/184486666 >> 172.21.15.164:0/184486666 conn(0x7fbab00d4d20 msgr2=0x7fbab00d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 -- 172.21.15.164:0/184486666 shutdown_connections 2024-04-26T20:25:40.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.539+0000 7fbab6000640 1 -- 172.21.15.164:0/184486666 wait complete. 2024-04-26T20:25:40.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.540+0000 7fbab6000640 1 Processor -- start 2024-04-26T20:25:40.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.541+0000 7fbab6000640 1 -- start start 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.541+0000 7fbab6000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 0x7fbab00ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.541+0000 7fbab6000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab01f8830 0x7fbab01f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab6000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0163310 con 0x7fbab01f8830 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab6000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbab00cf7f0 con 0x7fbab015ef30 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab6000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbab0163490 con 0x7fbab00d0600 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab4c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 0x7fbab00ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37492/0 (socket says 172.21.15.164:37492) 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbab4c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 0x7fbab00ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40558/0 (socket says 172.21.15.164:40558) 2024-04-26T20:25:40.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 -- 172.21.15.164:0/4135174913 learned_addr learned my addr 172.21.15.164:0/4135174913 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:40.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 -- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 msgr2=0x7fbab00ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 0x7fbab00ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 -- 172.21.15.164:0/4135174913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab01f8830 msgr2=0x7fbab01f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab01f8830 0x7fbab01f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.542+0000 7fbaafe00640 1 -- 172.21.15.164:0/4135174913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab00caea0 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaafe00640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba9c000c00 tx=0x7fba9c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c041020 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c0306c0 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c037040 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab00cb170 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab00c9a40 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba9c044020 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.543+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fba9c03dc40 con 0x7fbab015ef30 2024-04-26T20:25:40.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.544+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab00d4490 con 0x7fbab015ef30 2024-04-26T20:25:40.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.546+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbab00d4490 con 0x7fbab015ef30 2024-04-26T20:25:40.574 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:40.573+0000 7fc305d87040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:25:40.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.625+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "decfbdf4-6d09-495f-81a4-0935e7bae1b1", "id": 6} v 0) -- 0x7fbab00693b0 con 0x7fbab015ef30 2024-04-26T20:25:40.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.631+0000 7fbaacc00640 1 -- 172.21.15.164:0/4135174913 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "decfbdf4-6d09-495f-81a4-0935e7bae1b1", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fbab00693b0 con 0x7fbab015ef30 2024-04-26T20:25:40.632 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-04-26T20:25:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.632+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 msgr2=0x7fbab015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.632+0000 7fbab6000640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fba9c000c00 tx=0x7fba9c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.632+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 shutdown_connections 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.632+0000 7fbab6000640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbab01f8830 0x7fbab01f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.633+0000 7fbab6000640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbab015ef30 0x7fbab015b2b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.633+0000 7fbab6000640 1 --2- 172.21.15.164:0/4135174913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbab00d0600 0x7fbab00ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.633+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 >> 172.21.15.164:0/4135174913 conn(0x7fbab00d4d20 msgr2=0x7fbab00d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.633+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 shutdown_connections 2024-04-26T20:25:40.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.633+0000 7fbab6000640 1 -- 172.21.15.164:0/4135174913 wait complete. 2024-04-26T20:25:40.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 7bfe1def-9f02-4de8-b46f-2aeac197503f 7 2024-04-26T20:25:40.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.873+0000 7fb269c00640 1 Processor -- start 2024-04-26T20:25:40.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.873+0000 7fb269c00640 1 -- start start 2024-04-26T20:25:40.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.874+0000 7fb269c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 0x7fb2640d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb269c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2640b6800 con 0x7fb2640d2f70 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb269c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2640d0ff0 con 0x7fb2640b5bb0 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb269c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb2640d5ea0 con 0x7fb2640cc700 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb262000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb2640b5bb0 0x7fb2640cbff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42244/0 (socket says 172.21.15.164:42244) 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb262000640 1 -- 172.21.15.164:0/2026747575 learned_addr learned my addr 172.21.15.164:0/2026747575 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-04-26T20:25:40.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb263400640 1 --2- 172.21.15.164:0/2026747575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 0x7fb2640d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb261600640 1 -- 172.21.15.164:0/2026747575 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993419042 0 0) 0x7fb2640d0ff0 con 0x7fb2640b5bb0 2024-04-26T20:25:40.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.875+0000 7fb261600640 1 -- 172.21.15.164:0/2026747575 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb248003520 con 0x7fb2640b5bb0 2024-04-26T20:25:40.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.876+0000 7fb263400640 1 -- 172.21.15.164:0/2026747575 >> v1:172.21.15.164:6789/0 conn(0x7fb2640b5bb0 legacy=0x7fb2640cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.876+0000 7fb263400640 1 -- 172.21.15.164:0/2026747575 >> v1:172.21.15.43:6789/0 conn(0x7fb2640d2f70 legacy=0x7fb2640d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.876+0000 7fb263400640 1 -- 172.21.15.164:0/2026747575 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2640d5bc0 con 0x7fb2640cc700 2024-04-26T20:25:40.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb263400640 1 --2- 172.21.15.164:0/2026747575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 0x7fb2640d3350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009dc0 tx=0x7fb25c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa80e9f2e0fecc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb261600640 1 -- 172.21.15.164:0/2026747575 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c007740 con 0x7fb2640cc700 2024-04-26T20:25:40.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb261600640 1 -- 172.21.15.164:0/2026747575 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c0078e0 con 0x7fb2640cc700 2024-04-26T20:25:40.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb261600640 1 -- 172.21.15.164:0/2026747575 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c007cb0 con 0x7fb2640cc700 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb269c00640 1 -- 172.21.15.164:0/2026747575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 msgr2=0x7fb2640d3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.877+0000 7fb269c00640 1 --2- 172.21.15.164:0/2026747575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 0x7fb2640d3350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009dc0 tx=0x7fb25c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.878+0000 7fb269c00640 1 -- 172.21.15.164:0/2026747575 shutdown_connections 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.878+0000 7fb269c00640 1 --2- 172.21.15.164:0/2026747575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640cc700 0x7fb2640d3350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.878+0000 7fb269c00640 1 -- 172.21.15.164:0/2026747575 >> 172.21.15.164:0/2026747575 conn(0x7fb2640d4f20 msgr2=0x7fb2640d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.878+0000 7fb269c00640 1 -- 172.21.15.164:0/2026747575 shutdown_connections 2024-04-26T20:25:40.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.878+0000 7fb269c00640 1 -- 172.21.15.164:0/2026747575 wait complete. 2024-04-26T20:25:40.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.879+0000 7fb269c00640 1 Processor -- start 2024-04-26T20:25:40.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.880+0000 7fb269c00640 1 -- start start 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.880+0000 7fb269c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.880+0000 7fb269c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 0x7fb26409f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.880+0000 7fb269c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb269c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2640d3c40 con 0x7fb2640cc700 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb269c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2640d3f20 con 0x7fb2640b5bb0 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb269c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb2640b6100 con 0x7fb2640d2f70 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb263400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37500/0 (socket says 172.21.15.164:37500) 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb263400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:40578/0 (socket says 172.21.15.164:40578) 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 -- 172.21.15.164:0/3945792390 learned_addr learned my addr 172.21.15.164:0/3945792390 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-04-26T20:25:40.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262000640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 0x7fb26409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 -- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 msgr2=0x7fb2640c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 -- 172.21.15.164:0/3945792390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 msgr2=0x7fb26409f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 0x7fb26409f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262a00640 1 -- 172.21.15.164:0/3945792390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26415cb80 con 0x7fb2640b5bb0 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb263400640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.881+0000 7fb262000640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 0x7fb26409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb262a00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c3c0 tx=0x7fb25400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2540052d0 con 0x7fb2640b5bb0 2024-04-26T20:25:40.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb254005490 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254007260 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2640c8b80 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2640c9140 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb254007400 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.882+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb254003a00 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.883+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb264069a60 con 0x7fb2640b5bb0 2024-04-26T20:25:40.885 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:40.882+0000 7fc305d87040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:25:40.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.886+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb2640b6800 con 0x7fb2640b5bb0 2024-04-26T20:25:40.892 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:40.891+0000 7f381fb24040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:25:40.946 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:40.946+0000 7fc305d87040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:25:40.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.965+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7bfe1def-9f02-4de8-b46f-2aeac197503f", "id": 7} v 0) -- 0x7fb264074f70 con 0x7fb2640b5bb0 2024-04-26T20:25:40.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.974+0000 7fb25aa00640 1 -- 172.21.15.164:0/3945792390 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7bfe1def-9f02-4de8-b46f-2aeac197503f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb2540035f0 con 0x7fb2640b5bb0 2024-04-26T20:25:40.975 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 msgr2=0x7fb2640b9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c3c0 tx=0x7fb25400c880 comp rx=0 tx=0).stop 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 shutdown_connections 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2640d2f70 0x7fb2640c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2640cc700 0x7fb26409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 --2- 172.21.15.164:0/3945792390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2640b5bb0 0x7fb2640b9680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:40.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 >> 172.21.15.164:0/3945792390 conn(0x7fb2640d4f20 msgr2=0x7fb26415e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:40.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 shutdown_connections 2024-04-26T20:25:40.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:25:40.975+0000 7fb269c00640 1 -- 172.21.15.164:0/3945792390 wait complete. 2024-04-26T20:25:40.993 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T20:25:40.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T20:25:41.001 INFO:tasks.ceph.osd.0:Started 2024-04-26T20:25:41.001 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T20:25:41.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T20:25:41.010 INFO:tasks.ceph.osd.1:Started 2024-04-26T20:25:41.010 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T20:25:41.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T20:25:41.016 INFO:tasks.ceph.osd.2:Started 2024-04-26T20:25:41.016 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T20:25:41.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T20:25:41.022 INFO:tasks.ceph.osd.3:Started 2024-04-26T20:25:41.022 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T20:25:41.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T20:25:41.028 INFO:tasks.ceph.osd.4:Started 2024-04-26T20:25:41.028 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T20:25:41.028 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T20:25:41.033 INFO:tasks.ceph.osd.5:Started 2024-04-26T20:25:41.034 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T20:25:41.034 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T20:25:41.038 INFO:tasks.ceph.osd.6:Started 2024-04-26T20:25:41.039 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T20:25:41.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T20:25:41.044 INFO:tasks.ceph.osd.7:Started 2024-04-26T20:25:41.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:25:41.117 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.117+0000 7fc305d87040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:25:41.254 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:41.253+0000 7f381fb24040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:25:41.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.275+0000 7f2407800640 1 Processor -- start 2024-04-26T20:25:41.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.275+0000 7f2407800640 1 -- start start 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.276+0000 7f2407800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f240015f2d0 0x7f24000c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f2407800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24000d0cc0 con 0x7f24000c8eb0 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f2407800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24000d3a30 con 0x7f240015eef0 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f2407800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f24000c92e0 con 0x7f240015f2d0 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ffe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f240015f2d0 0x7f24000c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f2405800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f24000c8eb0 0x7f24000d4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50990/0 (socket says 172.21.15.43:50990) 2024-04-26T20:25:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f2405800640 1 -- 172.21.15.43:0/3003152508 learned_addr learned my addr 172.21.15.43:0/3003152508 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ffe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f240015f2d0 0x7f24000c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36600/0 (socket says 172.21.15.43:36600) 2024-04-26T20:25:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ffe00640 1 -- 172.21.15.43:0/3003152508 >> v1:172.21.15.164:6789/0 conn(0x7f240015eef0 legacy=0x7f24000d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ffe00640 1 -- 172.21.15.43:0/3003152508 >> v1:172.21.15.43:6789/0 conn(0x7f24000c8eb0 legacy=0x7f24000d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ffe00640 1 -- 172.21.15.43:0/3003152508 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000cfad0 con 0x7f240015f2d0 2024-04-26T20:25:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.277+0000 7f23ff400640 1 -- 172.21.15.43:0/3003152508 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2064253683 0 0) 0x7f24000d0cc0 con 0x7f24000c8eb0 2024-04-26T20:25:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.278+0000 7f23ffe00640 1 --2- 172.21.15.43:0/3003152508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f240015f2d0 0x7f24000c8970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f9135baceb1ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f23ff400640 1 -- 172.21.15.43:0/3003152508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec037070 con 0x7f240015f2d0 2024-04-26T20:25:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f23ff400640 1 -- 172.21.15.43:0/3003152508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ec032440 con 0x7f240015f2d0 2024-04-26T20:25:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f23ff400640 1 -- 172.21.15.43:0/3003152508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec032810 con 0x7f240015f2d0 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 -- 172.21.15.43:0/3003152508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f240015f2d0 msgr2=0x7f24000c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 --2- 172.21.15.43:0/3003152508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f240015f2d0 0x7f24000c8970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097a0 tx=0x7f23ec02a230 comp rx=0 tx=0).stop 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 -- 172.21.15.43:0/3003152508 shutdown_connections 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 --2- 172.21.15.43:0/3003152508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f240015f2d0 0x7f24000c8970 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 -- 172.21.15.43:0/3003152508 >> 172.21.15.43:0/3003152508 conn(0x7f240008e7e0 msgr2=0x7f240009e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 -- 172.21.15.43:0/3003152508 shutdown_connections 2024-04-26T20:25:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.279+0000 7f2407800640 1 -- 172.21.15.43:0/3003152508 wait complete. 2024-04-26T20:25:41.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.281+0000 7f2407800640 1 Processor -- start 2024-04-26T20:25:41.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.281+0000 7f2407800640 1 -- start start 2024-04-26T20:25:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f240015eef0 0x7f240015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f240015f2d0 0x7f240015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d3d10 con 0x7f24000c8eb0 2024-04-26T20:25:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24000d0090 con 0x7f240015f2d0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2407800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f24000d5170 con 0x7f240015eef0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2404e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2404e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36604/0 (socket says 172.21.15.43:36604) 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.282+0000 7f2404e00640 1 -- 172.21.15.43:0/159090474 learned_addr learned my addr 172.21.15.43:0/159090474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f23ffe00640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f240015eef0 0x7f240015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2405800640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f240015f2d0 0x7f240015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 -- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f240015eef0 msgr2=0x7f240015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f240015eef0 0x7f240015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 -- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f240015f2d0 msgr2=0x7f240015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f240015f2d0 0x7f240015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 -- 172.21.15.43:0/159090474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24000b8930 con 0x7f24000c8eb0 2024-04-26T20:25:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f2404e00640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c7c0 tx=0x7f23f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.283+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0003b70 con 0x7f24000c8eb0 2024-04-26T20:25:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f0003d30 con 0x7f24000c8eb0 2024-04-26T20:25:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24000b8650 con 0x7f24000c8eb0 2024-04-26T20:25:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000aa90 con 0x7f24000c8eb0 2024-04-26T20:25:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24000b8f40 con 0x7f24000c8eb0 2024-04-26T20:25:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f23f000ac30 con 0x7f24000c8eb0 2024-04-26T20:25:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.284+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f23f00057a0 con 0x7f24000c8eb0 2024-04-26T20:25:41.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.285+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2400069350 con 0x7f24000c8eb0 2024-04-26T20:25:41.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.289+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f23f00050d0 con 0x7f24000c8eb0 2024-04-26T20:25:41.316 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:41.316+0000 7f381fb24040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:25:41.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.354+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24000baa50 con 0x7f24000c8eb0 2024-04-26T20:25:41.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.355+0000 7f23fcc00640 1 -- 172.21.15.43:0/159090474 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f23f00052b0 con 0x7f24000c8eb0 2024-04-26T20:25:41.356 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:41.356 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:25:40.967754+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:25:41.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.356+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 msgr2=0x7f240014f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:41.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.356+0000 7f2407800640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c7c0 tx=0x7f23f000cc80 comp rx=0 tx=0).stop 2024-04-26T20:25:41.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 shutdown_connections 2024-04-26T20:25:41.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f240015eef0 0x7f240015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f240015f2d0 0x7f240015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 --2- 172.21.15.43:0/159090474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24000c8eb0 0x7f240014f5e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:41.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 >> 172.21.15.43:0/159090474 conn(0x7f240008e7e0 msgr2=0x7f24000d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:41.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 shutdown_connections 2024-04-26T20:25:41.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:41.357+0000 7f2407800640 1 -- 172.21.15.43:0/159090474 wait complete. 2024-04-26T20:25:41.374 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T20:25:41.374 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T20:25:41.447 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:41.447+0000 7f381fb24040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:25:41.508 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.508+0000 7fc305d87040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:25:41.566 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.566+0000 7fc305d87040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:25:41.705 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-26T20:25:41.704+0000 7fa819811640 -1 Falling back to public interface 2024-04-26T20:25:41.710 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-26T20:25:41.709+0000 7f2310328640 -1 Falling back to public interface 2024-04-26T20:25:41.716 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-26T20:25:41.715+0000 7fc7d1625640 -1 Falling back to public interface 2024-04-26T20:25:41.720 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-26T20:25:41.720+0000 7f78732ea640 -1 Falling back to public interface 2024-04-26T20:25:41.727 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-26T20:25:41.726+0000 7f97a06e3640 -1 Falling back to public interface 2024-04-26T20:25:41.737 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-26T20:25:41.736+0000 7fcbfffc4640 -1 Falling back to public interface 2024-04-26T20:25:41.742 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-26T20:25:41.742+0000 7f9b6cbd4640 -1 Falling back to public interface 2024-04-26T20:25:41.747 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-26T20:25:41.747+0000 7f788240c640 -1 Falling back to public interface 2024-04-26T20:25:41.758 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.758+0000 7fc305d87040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:25:41.794 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:41.794+0000 7f381fb24040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:25:41.850 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:41.849+0000 7f381fb24040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:25:41.934 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.933+0000 7fc305d87040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:25:41.994 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:41.994+0000 7fc305d87040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:25:42.033 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.032+0000 7f381fb24040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:25:42.079 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:42.078+0000 7fc305d87040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:25:42.134 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:42.133+0000 7fc305d87040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:25:42.179 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:25:42.200 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.199+0000 7f381fb24040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:25:42.258 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.257+0000 7f381fb24040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:25:42.341 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.340+0000 7f381fb24040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:25:42.342 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.341+0000 7f230a800640 1 Processor -- start 2024-04-26T20:25:42.342 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.342+0000 7f230a800640 1 -- start start 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f230a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23040babe0 0x7f23040cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f230a800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2304163750 con 0x7f230415f1a0 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f230a800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2304163a30 con 0x7f23040d0130 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f230a800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f23040d0580 con 0x7f23040babe0 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f2303400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f230415f1a0 0x7f2304162d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50992/0 (socket says 172.21.15.43:50992) 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.343+0000 7f2303400640 1 -- 172.21.15.43:0/1348819175 learned_addr learned my addr 172.21.15.43:0/1348819175 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:42.344 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302a00640 1 --2- 172.21.15.43:0/1348819175 >> v2:172.21.15.164:3300/0 conn(0x7f23040babe0 0x7f23040cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78308454 0 0) 0x7f2304163750 con 0x7f230415f1a0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22e8003520 con 0x7f230415f1a0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2195193397 0 0) 0x7f22e8003520 con 0x7f230415f1a0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 >> v2:172.21.15.164:3300/0 conn(0x7f23040babe0 msgr2=0x7f23040cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 --2- 172.21.15.43:0/1348819175 >> v2:172.21.15.164:3300/0 conn(0x7f23040babe0 0x7f23040cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 >> v1:172.21.15.164:6789/0 conn(0x7f23040d0130 legacy=0x7f23040d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040d31d0 con 0x7f230415f1a0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f22f4001670 con 0x7f230415f1a0 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 >> v1:172.21.15.43:6789/0 conn(0x7f230415f1a0 legacy=0x7f2304162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8003a00 0x7f22e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 0x7f22e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22e8005210 0x7f22e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.347 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f23040d0580 con 0x7f22e8004800 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22e800c160 con 0x7f22e8003a00 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.344+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22e800c360 con 0x7f22e8005210 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 0x7f22e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2303400640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8003a00 0x7f22e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 -- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22e8005210 msgr2=0x7f22e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2303e00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22e8005210 0x7f22e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22e8005210 0x7f22e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 -- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8003a00 msgr2=0x7f22e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8003a00 0x7f22e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2302a00640 1 -- 172.21.15.43:0/1348819175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e8003520 con 0x7f22e8004800 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2303e00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22e8005210 0x7f22e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.345+0000 7f2303e00640 1 -- 172.21.15.43:0/1348819175 reap_dead start 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.346+0000 7f2302a00640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 0x7f22e8004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009a90 tx=0x7f22e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b082d914faeb4151 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:42.348 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.346+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0036070 con 0x7f22e8004800 2024-04-26T20:25:42.349 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.346+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0031400 con 0x7f22e8004800 2024-04-26T20:25:42.349 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.346+0000 7f2302000640 1 -- 172.21.15.43:0/1348819175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0009690 con 0x7f22e8004800 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 -- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 msgr2=0x7f22e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 0x7f22e8004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009a90 tx=0x7f22e8005830 comp rx=0 tx=0).stop 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 -- 172.21.15.43:0/1348819175 shutdown_connections 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 --2- 172.21.15.43:0/1348819175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22e8004800 0x7f22e8004c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 -- 172.21.15.43:0/1348819175 >> 172.21.15.43:0/1348819175 conn(0x7f2304073b60 msgr2=0x7f2304073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 -- 172.21.15.43:0/1348819175 shutdown_connections 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.347+0000 7f230a800640 1 -- 172.21.15.43:0/1348819175 wait complete. 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.349+0000 7f230a800640 1 Processor -- start 2024-04-26T20:25:42.350 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.349+0000 7f230a800640 1 -- start start 2024-04-26T20:25:42.352 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 0x7f23041f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 0x7f230415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f230415f580 con 0x7f23041f9170 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23040d2f80 con 0x7f22e8004800 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.350+0000 7f230a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f23040cfbf0 con 0x7f23040c88f0 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36620/0 (socket says 172.21.15.43:36620) 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 -- 172.21.15.43:0/2141160093 learned_addr learned my addr 172.21.15.43:0/2141160093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2303400640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 0x7f23041f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2303e00640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 0x7f230415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 -- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 msgr2=0x7f230415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 0x7f230415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 -- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 msgr2=0x7f23041f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.353 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 0x7f23041f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 -- 172.21.15.43:0/2141160093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23040c9750 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2303e00640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 0x7f230415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2303400640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 0x7f23041f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.351+0000 7f2302a00640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22f0034040 tx=0x7f22f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f003f030 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0037040 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0036070 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23040c8eb0 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23040c9470 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22f0042070 con 0x7f23041f9170 2024-04-26T20:25:42.354 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.352+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f22f0009690 con 0x7f23041f9170 2024-04-26T20:25:42.355 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.353+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2304069c80 con 0x7f23041f9170 2024-04-26T20:25:42.355 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.355+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f22f0044c30 con 0x7f23041f9170 2024-04-26T20:25:42.395 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.394+0000 7f381fb24040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:25:42.422 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.422+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23040d2b30 con 0x7f23041f9170 2024-04-26T20:25:42.423 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.423+0000 7f22fb400640 1 -- 172.21.15.43:0/2141160093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f22f0044e10 con 0x7f23041f9170 2024-04-26T20:25:42.424 INFO:teuthology.misc.health.smithi043.stdout: 2024-04-26T20:25:42.424 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:25:40.967754+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 msgr2=0x7f230415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22f0034040 tx=0x7f22f0004730 comp rx=0 tx=0).stop 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 shutdown_connections 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f23040c88f0 0x7f230415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23041f9170 0x7f230415b6c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 --2- 172.21.15.43:0/2141160093 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e8004800 0x7f23041f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 >> 172.21.15.43:0/2141160093 conn(0x7f2304073b60 msgr2=0x7f230415cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.424+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 shutdown_connections 2024-04-26T20:25:42.425 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:42.425+0000 7f230a800640 1 -- 172.21.15.43:0/2141160093 wait complete. 2024-04-26T20:25:42.437 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:25:42.702 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:42.701+0000 7fc305d87040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:25:42.838 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:42.838+0000 7fc305d87040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:25:42.947 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:42.946+0000 7f381fb24040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:25:43.078 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-26T20:25:43.077+0000 7f381fb24040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:25:43.879 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:43.878+0000 7fc2a6400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T20:25:43.880 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-26T20:25:43.880+0000 7fc2a6400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T20:25:45.128 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-26T20:25:45.128+0000 7f788240c640 -1 osd.2 0 log_to_monitors true 2024-04-26T20:25:45.141 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-26T20:25:45.140+0000 7fcbfffc4640 -1 osd.6 0 log_to_monitors true 2024-04-26T20:25:45.302 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-26T20:25:45.301+0000 7f78732ea640 -1 osd.1 0 log_to_monitors true 2024-04-26T20:25:45.306 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-26T20:25:45.305+0000 7f2310328640 -1 osd.4 0 log_to_monitors true 2024-04-26T20:25:45.314 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-26T20:25:45.313+0000 7fa819811640 -1 osd.5 0 log_to_monitors true 2024-04-26T20:25:45.322 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-26T20:25:45.322+0000 7fc7d1625640 -1 osd.0 0 log_to_monitors true 2024-04-26T20:25:45.374 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-26T20:25:45.373+0000 7f9b6cbd4640 -1 osd.7 0 log_to_monitors true 2024-04-26T20:25:45.579 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-26T20:25:45.578+0000 7f97a06e3640 -1 osd.3 0 log_to_monitors true 2024-04-26T20:25:48.626 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-26T20:25:48.624+0000 7fc7cc000640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T20:25:48.632 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-26T20:25:48.632+0000 7fc7c2000640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:48.637 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-26T20:25:48.637+0000 7f786e200640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T20:25:48.642 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-26T20:25:48.642+0000 7f7864200640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:48.653 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-26T20:25:48.653+0000 7f787cc00640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T20:25:48.661 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-26T20:25:48.661+0000 7f7872c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:48.680 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-26T20:25:48.680+0000 7f979b600640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T20:25:48.689 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-26T20:25:48.689+0000 7f9791600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:49.221 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-26T20:25:49.221+0000 7f9b68400640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T20:25:49.226 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-26T20:25:49.226+0000 7f9b5da00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:49.242 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:25:49.248 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-26T20:25:49.248+0000 7fa814000640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T20:25:49.257 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-26T20:25:49.257+0000 7fa80a000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:49.292 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-26T20:25:49.291+0000 7f230b200640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T20:25:49.300 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-26T20:25:49.299+0000 7f2301200640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:49.325 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-26T20:25:49.325+0000 7fcbfb800640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T20:25:49.329 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-26T20:25:49.329+0000 7fcbf0e00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:25:49.426 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.425+0000 7f67c8600640 1 Processor -- start 2024-04-26T20:25:49.426 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.426+0000 7f67c8600640 1 -- start start 2024-04-26T20:25:49.428 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c8600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67c00babe0 0x7f67c00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.428 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c8600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67c00d0130 0x7f67c00d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.428 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c8600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c0163750 con 0x7f67c015f1a0 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c8600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f67c00d0580 con 0x7f67c00d0130 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c8600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f67c00d2f80 con 0x7f67c00babe0 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c5c00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f67c015f1a0 0x7f67c0162d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41714/0 (socket says 172.21.15.43:41714) 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c5c00640 1 -- 172.21.15.43:0/232225149 learned_addr learned my addr 172.21.15.43:0/232225149 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.427+0000 7f67c6600640 1 --2- 172.21.15.43:0/232225149 >> v2:172.21.15.43:3300/0 conn(0x7f67c00d0130 0x7f67c00d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67c5200640 1 --2- 172.21.15.43:0/232225149 >> v2:172.21.15.164:3300/0 conn(0x7f67c00babe0 0x7f67c00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67c6600640 1 -- 172.21.15.43:0/232225149 >> v2:172.21.15.164:3300/0 conn(0x7f67c00babe0 msgr2=0x7f67c00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67c6600640 1 --2- 172.21.15.43:0/232225149 >> v2:172.21.15.164:3300/0 conn(0x7f67c00babe0 0x7f67c00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67c6600640 1 -- 172.21.15.43:0/232225149 >> v1:172.21.15.43:6789/0 conn(0x7f67c015f1a0 legacy=0x7f67c0162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67c6600640 1 -- 172.21.15.43:0/232225149 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c0163a30 con 0x7f67c00d0130 2024-04-26T20:25:49.429 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.428+0000 7f67bfe00640 1 -- 172.21.15.43:0/232225149 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577793956 0 0) 0x7f67c0163750 con 0x7f67c015f1a0 2024-04-26T20:25:49.430 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.429+0000 7f67c6600640 1 --2- 172.21.15.43:0/232225149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c00d0130 0x7f67c00d39d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67c0074ae0 tx=0x7f67b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b93b3ff8dea59a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.429+0000 7f67bfe00640 1 -- 172.21.15.43:0/232225149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8037070 con 0x7f67c00d0130 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.429+0000 7f67bfe00640 1 -- 172.21.15.43:0/232225149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8032440 con 0x7f67c00d0130 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.429+0000 7f67bfe00640 1 -- 172.21.15.43:0/232225149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b802ae60 con 0x7f67c00d0130 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 -- 172.21.15.43:0/232225149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c00d0130 msgr2=0x7f67c00d39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 --2- 172.21.15.43:0/232225149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c00d0130 0x7f67c00d39d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67c0074ae0 tx=0x7f67b802a110 comp rx=0 tx=0).stop 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 -- 172.21.15.43:0/232225149 shutdown_connections 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 --2- 172.21.15.43:0/232225149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c00d0130 0x7f67c00d39d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 --2- 172.21.15.43:0/232225149 >> v2:172.21.15.164:3300/0 conn(0x7f67c00babe0 0x7f67c00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 -- 172.21.15.43:0/232225149 >> 172.21.15.43:0/232225149 conn(0x7f67c0073b60 msgr2=0x7f67c0073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 -- 172.21.15.43:0/232225149 shutdown_connections 2024-04-26T20:25:49.431 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.430+0000 7f67c8600640 1 -- 172.21.15.43:0/232225149 wait complete. 2024-04-26T20:25:49.432 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.432+0000 7f67c8600640 1 Processor -- start 2024-04-26T20:25:49.432 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.432+0000 7f67c8600640 1 -- start start 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67c00babe0 0x7f67c00c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c00d0130 0x7f67c00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67c015f580 con 0x7f67c015f1a0 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67c00b7fc0 con 0x7f67c00d0130 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c8600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f67c0163410 con 0x7f67c00babe0 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53260/0 (socket says 172.21.15.43:53260) 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 -- 172.21.15.43:0/1570855730 learned_addr learned my addr 172.21.15.43:0/1570855730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 -- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67c00babe0 msgr2=0x7f67c00c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.434 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67c00babe0 0x7f67c00c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.435 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 -- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c00d0130 msgr2=0x7f67c00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.435 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c00d0130 0x7f67c00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.435 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.433+0000 7f67c6600640 1 -- 172.21.15.43:0/1570855730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c0155e30 con 0x7f67c015f1a0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67c6600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009890 tx=0x7f67b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8037050 con 0x7f67c015f1a0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8038040 con 0x7f67c015f1a0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c0156110 con 0x7f67c015f1a0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8006e10 con 0x7f67c015f1a0 2024-04-26T20:25:49.436 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.434+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c00c7f40 con 0x7f67c015f1a0 2024-04-26T20:25:49.437 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.435+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c0069c80 con 0x7f67c015f1a0 2024-04-26T20:25:49.437 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.436+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f67b8030980 con 0x7f67c015f1a0 2024-04-26T20:25:49.437 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.437+0000 7f67bd600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 0x7f67a006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.438 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.437+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f67b80ab7a0 con 0x7f67c015f1a0 2024-04-26T20:25:49.438 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.438+0000 7f67c5c00640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 0x7f67a006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.438 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.438+0000 7f67c5c00640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 0x7f67a006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007fd0 tx=0x7f67b0007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.443 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.443+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67b8032e20 con 0x7f67c015f1a0 2024-04-26T20:25:49.640 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.640+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67c00b8530 con 0x7f67c015f1a0 2024-04-26T20:25:49.652 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.652+0000 7f67bd600640 1 -- 172.21.15.43:0/1570855730 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10885 (secure 0 0 0) 0x7f67b807aa30 con 0x7f67c015f1a0 2024-04-26T20:25:49.653 INFO:teuthology.misc.health.smithi043.stdout: 2024-04-26T20:25:49.653 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":13,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:25:49.633232+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:25:49.658 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.657+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 msgr2=0x7f67a006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.658 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.657+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 0x7f67a006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007fd0 tx=0x7f67b0007960 comp rx=0 tx=0).stop 2024-04-26T20:25:49.658 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.657+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 msgr2=0x7f67c00ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.657+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009890 tx=0x7f67b80046c0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 shutdown_connections 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f67a006c860 0x7f67a006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67c015f1a0 0x7f67c00ce6a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67c00d0130 0x7f67c00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 --2- 172.21.15.43:0/1570855730 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67c00babe0 0x7f67c00c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 >> 172.21.15.43:0/1570855730 conn(0x7f67c0073b60 msgr2=0x7f67c00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 shutdown_connections 2024-04-26T20:25:49.659 INFO:teuthology.misc.health.smithi043.stderr:2024-04-26T20:25:49.658+0000 7f67c8600640 1 -- 172.21.15.43:0/1570855730 wait complete. 2024-04-26T20:25:49.673 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T20:25:49.673 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T20:25:49.674 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:25:49.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:25:49.678 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:25:49.678 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:25:49.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:25:49.682 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:25:49.683 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:25:49.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:25:49.690 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:25:49.690 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:25:49.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:25:49.696 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:25:49.697 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T20:25:49.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:25:49.814 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-26T20:25:49.815 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-26T20:25:49.839 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-04-26T20:25:49.840 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-04-26T20:25:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.864+0000 7f3ebda00640 1 Processor -- start 2024-04-26T20:25:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.864+0000 7f3ebda00640 1 -- start start 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3eb815cdb0 0x7f3eb80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb80ce590 0x7f3eb80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 0x7f3eb80cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb80d14d0 con 0x7f3eb815cdb0 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb80af770 con 0x7f3eb80d1070 2024-04-26T20:25:49.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebda00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb80af9c0 con 0x7f3eb80ce590 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3eb7e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3eb815cdb0 0x7f3eb80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3eb7e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3eb815cdb0 0x7f3eb80ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53274/0 (socket says 172.21.15.43:53274) 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3eb7e00640 1 -- 172.21.15.43:0/2703385513 learned_addr learned my addr 172.21.15.43:0/2703385513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3eafe00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb80ce590 0x7f3eb80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.865+0000 7f3ebd000640 1 --2- 172.21.15.43:0/2703385513 >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 0x7f3eb80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3eb7e00640 1 -- 172.21.15.43:0/2703385513 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb80ce590 msgr2=0x7f3eb80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb80ce590 0x7f3eb80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3eb7e00640 1 -- 172.21.15.43:0/2703385513 >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 msgr2=0x7f3eb80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2703385513 >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 0x7f3eb80cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3eb7e00640 1 -- 172.21.15.43:0/2703385513 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80afca0 con 0x7f3eb815cdb0 2024-04-26T20:25:49.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.866+0000 7f3ebd000640 1 --2- 172.21.15.43:0/2703385513 >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 0x7f3eb80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.867+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb815cdb0 0x7f3eb80ce050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80739a0 tx=0x7f3ea802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163eb6a485975185 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.867+0000 7f3eb7400640 1 -- 172.21.15.43:0/2703385513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8036070 con 0x7f3eb815cdb0 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.867+0000 7f3eb7400640 1 -- 172.21.15.43:0/2703385513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea8031600 con 0x7f3eb815cdb0 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.867+0000 7f3eb7400640 1 -- 172.21.15.43:0/2703385513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80319d0 con 0x7f3eb815cdb0 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 -- 172.21.15.43:0/2703385513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb815cdb0 msgr2=0x7f3eb80ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb815cdb0 0x7f3eb80ce050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80739a0 tx=0x7f3ea802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 -- 172.21.15.43:0/2703385513 shutdown_connections 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2703385513 >> v2:172.21.15.164:3300/0 conn(0x7f3eb80d1070 0x7f3eb80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb80ce590 0x7f3eb80d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2703385513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb815cdb0 0x7f3eb80ce050 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 -- 172.21.15.43:0/2703385513 >> 172.21.15.43:0/2703385513 conn(0x7f3eb80d29a0 msgr2=0x7f3eb80d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 -- 172.21.15.43:0/2703385513 shutdown_connections 2024-04-26T20:25:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.868+0000 7f3ebda00640 1 -- 172.21.15.43:0/2703385513 wait complete. 2024-04-26T20:25:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.869+0000 7f3ebda00640 1 Processor -- start 2024-04-26T20:25:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.869+0000 7f3ebda00640 1 -- start start 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 0x7f3eb815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb815cdb0 0x7f3eb80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb815d190 con 0x7f3eb80d1070 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80b0260 con 0x7f3eb80ce590 2024-04-26T20:25:49.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.870+0000 7f3ebda00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb80d0730 con 0x7f3eb815cdb0 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53276/0 (socket says 172.21.15.43:53276) 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 -- 172.21.15.43:0/2172602699 learned_addr learned my addr 172.21.15.43:0/2172602699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 0x7f3eb815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3ebd000640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb815cdb0 0x7f3eb80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 -- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb815cdb0 msgr2=0x7f3eb80cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb815cdb0 0x7f3eb80cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 -- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 msgr2=0x7f3eb815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 0x7f3eb815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 -- 172.21.15.43:0/2172602699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80b5760 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 0x7f3eb815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eafe00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e890 tx=0x7f3ea000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.871+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0017070 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.872+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea000c940 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.872+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000cd10 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.872+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb80b5a40 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.872+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb80b6000 con 0x7f3eb80d1070 2024-04-26T20:25:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.873+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8069350 con 0x7f3eb80d1070 2024-04-26T20:25:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.874+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f3ea001a070 con 0x7f3eb80d1070 2024-04-26T20:25:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.875+0000 7f3eb4c00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 0x7f3e9406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.875+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f3ea008b970 con 0x7f3eb80d1070 2024-04-26T20:25:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.875+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 0x7f3e9406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.876+0000 7f3eb7e00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 0x7f3e9406ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8033000 tx=0x7f3ea8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:49.880+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ea005a2b0 con 0x7f3eb80d1070 2024-04-26T20:25:50.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.080+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eb80b4640 con 0x7f3eb80d1070 2024-04-26T20:25:50.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.082+0000 7f3eb4c00640 1 -- 172.21.15.43:0/2172602699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10885 (secure 0 0 0) 0x7f3ea0059bb0 con 0x7f3eb80d1070 2024-04-26T20:25:50.083 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:50.083 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:25:49.633232+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 msgr2=0x7f3e9406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 0x7f3e9406ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8033000 tx=0x7f3ea8009000 comp rx=0 tx=0).stop 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 msgr2=0x7f3eb80ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e890 tx=0x7f3ea000ed50 comp rx=0 tx=0).stop 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 shutdown_connections 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3e9406c860 0x7f3e9406ed20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3eb815cdb0 0x7f3eb80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3eb80d1070 0x7f3eb80ca7d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 --2- 172.21.15.43:0/2172602699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3eb80ce590 0x7f3eb815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 >> 172.21.15.43:0/2172602699 conn(0x7f3eb80d29a0 msgr2=0x7f3eb806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 shutdown_connections 2024-04-26T20:25:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.084+0000 7f3ebda00640 1 -- 172.21.15.43:0/2172602699 wait complete. 2024-04-26T20:25:50.102 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-26T20:25:50.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:25:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.302+0000 7ff9e3200640 1 Processor -- start 2024-04-26T20:25:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.302+0000 7ff9e3200640 1 -- start start 2024-04-26T20:25:50.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 0x7ff9e40cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e3200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e4072db0 con 0x7ff9e415bb90 2024-04-26T20:25:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e40cee40 con 0x7ff9e40af710 2024-04-26T20:25:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e40cec10 con 0x7ff9e40afb70 2024-04-26T20:25:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e1e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff9e415bb90 0x7ff9e40737a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41742/0 (socket says 172.21.15.43:41742) 2024-04-26T20:25:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.304+0000 7ff9e1e00640 1 -- 172.21.15.43:0/178761546 learned_addr learned my addr 172.21.15.43:0/178761546 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e2800640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 0x7ff9e40cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607634687 0 0) 0x7ff9e4072db0 con 0x7ff9e415bb90 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c8003520 con 0x7ff9e415bb90 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 528617098 0 0) 0x7ff9c8003520 con 0x7ff9e415bb90 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 msgr2=0x7ff9e40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 0x7ff9e40cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 >> v1:172.21.15.164:6789/0 conn(0x7ff9e40af710 legacy=0x7ff9e40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e40727f0 con 0x7ff9e415bb90 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7ff9d4001670 con 0x7ff9e415bb90 2024-04-26T20:25:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 >> v1:172.21.15.43:6789/0 conn(0x7ff9e415bb90 legacy=0x7ff9e40737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c80039e0 0x7ff9c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c8004800 0x7ff9c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 0x7ff9c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c800ba70 con 0x7ff9c80039e0 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c800bc20 con 0x7ff9c8004800 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.305+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c800be20 con 0x7ff9e40afb70 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c80039e0 0x7ff9c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1400640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c8004800 0x7ff9c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 -- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 msgr2=0x7ff9c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9e40afb70 0x7ff9c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 -- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c8004800 msgr2=0x7ff9c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9c8004800 0x7ff9c8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.306+0000 7ff9e1e00640 1 -- 172.21.15.43:0/178761546 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c8003520 con 0x7ff9c80039e0 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.307+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c80039e0 0x7ff9c8003e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c210 tx=0x7ff9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36d0e870bf7fcd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.307+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4007d10 con 0x7ff9c80039e0 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.307+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4005ad0 con 0x7ff9c80039e0 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.307+0000 7ff9e0a00640 1 -- 172.21.15.43:0/178761546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4007070 con 0x7ff9c80039e0 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 -- 172.21.15.43:0/178761546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c80039e0 msgr2=0x7ff9c8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 --2- 172.21.15.43:0/178761546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c80039e0 0x7ff9c8003e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c210 tx=0x7ff9c8005830 comp rx=0 tx=0).stop 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e2800640 1 -- 172.21.15.43:0/178761546 reap_dead start 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 -- 172.21.15.43:0/178761546 shutdown_connections 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 -- 172.21.15.43:0/178761546 >> 172.21.15.43:0/178761546 conn(0x7ff9e4068090 msgr2=0x7ff9e4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 -- 172.21.15.43:0/178761546 shutdown_connections 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.308+0000 7ff9e3200640 1 -- 172.21.15.43:0/178761546 wait complete. 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.310+0000 7ff9e3200640 1 Processor -- start 2024-04-26T20:25:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.310+0000 7ff9e3200640 1 -- start start 2024-04-26T20:25:50.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9c80039e0 0x7ff9e41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 0x7ff9e40aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4073eb0 con 0x7ff9e415c980 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e40725a0 con 0x7ff9e40ae8c0 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e40cf0e0 con 0x7ff9c80039e0 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53294/0 (socket says 172.21.15.43:53294) 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 -- 172.21.15.43:0/3877940870 learned_addr learned my addr 172.21.15.43:0/3877940870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9c80039e0 0x7ff9e41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e2800640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 0x7ff9e40aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 -- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9c80039e0 msgr2=0x7ff9e41545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9c80039e0 0x7ff9e41545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 -- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 msgr2=0x7ff9e40aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 0x7ff9e40aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.311+0000 7ff9e1400640 1 -- 172.21.15.43:0/3877940870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e41f5110 con 0x7ff9e415c980 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9e2800640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 0x7ff9e40aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9e1400640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000d5d0 tx=0x7ff9d000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0003610 con 0x7ff9e415c980 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d0017070 con 0x7ff9e415c980 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e41f53f0 con 0x7ff9e415c980 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00039f0 con 0x7ff9e415c980 2024-04-26T20:25:50.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.312+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e41f59b0 con 0x7ff9e415c980 2024-04-26T20:25:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.314+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e4069350 con 0x7ff9e415c980 2024-04-26T20:25:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.314+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff9d0003b90 con 0x7ff9e415c980 2024-04-26T20:25:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.316+0000 7ff9da000640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 0x7ff9c406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.316+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7ff9d008b370 con 0x7ff9e415c980 2024-04-26T20:25:50.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.316+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 0x7ff9c406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:50.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.317+0000 7ff9e1e00640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 0x7ff9c406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4032000 tx=0x7ff9d402d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:50.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.320+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9d0059ea0 con 0x7ff9e415c980 2024-04-26T20:25:50.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.564+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9e415f380 con 0x7ff9e415c980 2024-04-26T20:25:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.565+0000 7ff9da000640 1 -- 172.21.15.43:0/3877940870 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7ff9d00597a0 con 0x7ff9e415c980 2024-04-26T20:25:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:50.566 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:25:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-04-26T20:25:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 msgr2=0x7ff9c406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 0x7ff9c406ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4032000 tx=0x7ff9d402d000 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 msgr2=0x7ff9e415ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000d5d0 tx=0x7ff9d000da90 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 shutdown_connections 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff9c406c860 0x7ff9c406ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9e40ae8c0 0x7ff9e40aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e415c980 0x7ff9e415ce00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 --2- 172.21.15.43:0/3877940870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9c80039e0 0x7ff9e41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 >> 172.21.15.43:0/3877940870 conn(0x7ff9e4068090 msgr2=0x7ff9e40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 shutdown_connections 2024-04-26T20:25:50.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:50.568+0000 7ff9e3200640 1 -- 172.21.15.43:0/3877940870 wait complete. 2024-04-26T20:25:52.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:25:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.806+0000 7ff71a600640 1 Processor -- start 2024-04-26T20:25:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.806+0000 7ff71a600640 1 -- start start 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.807+0000 7ff71a600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff71415e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.807+0000 7ff71a600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7140c8830 con 0x7ff7140b71a0 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.807+0000 7ff71a600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7140d3cb0 con 0x7ff71415ef00 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.807+0000 7ff71a600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff7140c8af0 con 0x7ff7140b7580 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.808+0000 7ff713400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7140b71a0 0x7ff7140d34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41744/0 (socket says 172.21.15.43:41744) 2024-04-26T20:25:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.808+0000 7ff713400640 1 -- 172.21.15.43:0/77625018 learned_addr learned my addr 172.21.15.43:0/77625018 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.808+0000 7ff712a00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff71415e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.808+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810790361 0 0) 0x7ff7140c8830 con 0x7ff7140b71a0 2024-04-26T20:25:52.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.808+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f8003520 con 0x7ff7140b71a0 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2491507728 0 0) 0x7ff6f8003520 con 0x7ff7140b71a0 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 msgr2=0x7ff71415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff71415e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 >> v1:172.21.15.164:6789/0 conn(0x7ff71415ef00 legacy=0x7ff7140cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7140c8dd0 con 0x7ff7140b71a0 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7ff7040019f0 con 0x7ff7140b71a0 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 >> v1:172.21.15.43:6789/0 conn(0x7ff7140b71a0 legacy=0x7ff7140d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f80039e0 0x7ff6f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f8004390 0x7ff6f8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff6f8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140c8af0 con 0x7ff6f8004390 2024-04-26T20:25:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f800ba70 con 0x7ff6f80039e0 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f800bc70 con 0x7ff7140b7580 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff712a00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f8004390 0x7ff6f8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.809+0000 7ff713400640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f80039e0 0x7ff6f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff713e00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff6f8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff712a00640 1 -- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 msgr2=0x7ff6f8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff712a00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff7140b7580 0x7ff6f8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff712a00640 1 -- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f80039e0 msgr2=0x7ff6f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff712a00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f80039e0 0x7ff6f8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.810+0000 7ff712a00640 1 -- 172.21.15.43:0/77625018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f8003520 con 0x7ff6f8004390 2024-04-26T20:25:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.811+0000 7ff712a00640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f8004390 0x7ff6f8004860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff71408d1d0 tx=0x7ff6f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44e771b25a3f488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700036070 con 0x7ff6f8004390 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff700031400 con 0x7ff6f8004390 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff712000640 1 -- 172.21.15.43:0/77625018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700009690 con 0x7ff6f8004390 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff71a600640 1 -- 172.21.15.43:0/77625018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f8004390 msgr2=0x7ff6f8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff71a600640 1 --2- 172.21.15.43:0/77625018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f8004390 0x7ff6f8004860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff71408d1d0 tx=0x7ff6f8005830 comp rx=0 tx=0).stop 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff713e00640 1 -- 172.21.15.43:0/77625018 reap_dead start 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff71a600640 1 -- 172.21.15.43:0/77625018 shutdown_connections 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.812+0000 7ff71a600640 1 -- 172.21.15.43:0/77625018 >> 172.21.15.43:0/77625018 conn(0x7ff7140d4d70 msgr2=0x7ff7140d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.813+0000 7ff71a600640 1 -- 172.21.15.43:0/77625018 shutdown_connections 2024-04-26T20:25:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.813+0000 7ff71a600640 1 -- 172.21.15.43:0/77625018 wait complete. 2024-04-26T20:25:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.815+0000 7ff71a600640 1 Processor -- start 2024-04-26T20:25:52.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.816+0000 7ff71a600640 1 -- start start 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f8004390 0x7ff714153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff714153dd0 0x7ff714154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140ca760 con 0x7ff7140b94b0 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7140ca5e0 con 0x7ff6f8004390 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff71a600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff7140ca8e0 con 0x7ff714153dd0 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff712a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff712a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53338/0 (socket says 172.21.15.43:53338) 2024-04-26T20:25:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff712a00640 1 -- 172.21.15.43:0/2009695425 learned_addr learned my addr 172.21.15.43:0/2009695425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff713e00640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff714153dd0 0x7ff714154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.817+0000 7ff713400640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f8004390 0x7ff714153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 -- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff714153dd0 msgr2=0x7ff714154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff714153dd0 0x7ff714154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 -- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f8004390 msgr2=0x7ff714153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f8004390 0x7ff714153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 -- 172.21.15.43:0/2009695425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff714158900 con 0x7ff7140b94b0 2024-04-26T20:25:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff712a00640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff700034040 tx=0x7ff700004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700040020 con 0x7ff7140b94b0 2024-04-26T20:25:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70003b6a0 con 0x7ff7140b94b0 2024-04-26T20:25:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7141549b0 con 0x7ff7140b94b0 2024-04-26T20:25:52.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700036040 con 0x7ff7140b94b0 2024-04-26T20:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.818+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff714154f70 con 0x7ff7140b94b0 2024-04-26T20:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.820+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff714069350 con 0x7ff7140b94b0 2024-04-26T20:25:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.821+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff700043020 con 0x7ff7140b94b0 2024-04-26T20:25:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.822+0000 7ff70b400640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 0x7ff6f406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.822+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7ff7000abf80 con 0x7ff7140b94b0 2024-04-26T20:25:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.823+0000 7ff713400640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 0x7ff6f406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.823+0000 7ff713400640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 0x7ff6f406ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff704001ff0 tx=0x7ff704009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:52.828+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff70004a030 con 0x7ff7140b94b0 2024-04-26T20:25:53.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.087+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7140c8830 con 0x7ff7140b94b0 2024-04-26T20:25:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.088+0000 7ff70b400640 1 -- 172.21.15.43:0/2009695425 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7ff70007a6b0 con 0x7ff7140b94b0 2024-04-26T20:25:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:53.089 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:25:53.089 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-04-26T20:25:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 msgr2=0x7ff6f406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 0x7ff6f406ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff704001ff0 tx=0x7ff704009040 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 msgr2=0x7ff7140b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff700034040 tx=0x7ff700004730 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 shutdown_connections 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff6f406c860 0x7ff6f406ed20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff714153dd0 0x7ff714154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7140b94b0 0x7ff7140b9930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 --2- 172.21.15.43:0/2009695425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6f8004390 0x7ff714153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 >> 172.21.15.43:0/2009695425 conn(0x7ff7140d4d70 msgr2=0x7ff71415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 shutdown_connections 2024-04-26T20:25:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.091+0000 7ff71a600640 1 -- 172.21.15.43:0/2009695425 wait complete. 2024-04-26T20:25:53.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:25:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.298+0000 7f0200200640 1 Processor -- start 2024-04-26T20:25:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.299+0000 7f0200200640 1 -- start start 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f0200200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f81467c0 0x7f01f8072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f0200200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 0x7f01f80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f0200200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f8146ba0 con 0x7f01f80b4140 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f0200200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f01f8073dd0 con 0x7f01f81467c0 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f0200200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f01f80731c0 con 0x7f01f80b3ce0 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f01fd800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f81467c0 0x7f01f8072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f01fd800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f81467c0 0x7f01f8072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53352/0 (socket says 172.21.15.43:53352) 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f01fd800640 1 -- 172.21.15.43:0/858746481 learned_addr learned my addr 172.21.15.43:0/858746481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.300+0000 7f01fce00640 1 --2- 172.21.15.43:0/858746481 >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 0x7f01f80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.301+0000 7f01fd800640 1 -- 172.21.15.43:0/858746481 >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 msgr2=0x7f01f80d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.301+0000 7f01fd800640 1 --2- 172.21.15.43:0/858746481 >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 0x7f01f80d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.301+0000 7f01fd800640 1 -- 172.21.15.43:0/858746481 >> v1:172.21.15.164:6789/0 conn(0x7f01f80b4140 legacy=0x7f01f8160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.301+0000 7f01fd800640 1 -- 172.21.15.43:0/858746481 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f80734a0 con 0x7f01f81467c0 2024-04-26T20:25:53.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.301+0000 7f01fce00640 1 --2- 172.21.15.43:0/858746481 >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 0x7f01f80d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:53.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.302+0000 7f01fd800640 1 --2- 172.21.15.43:0/858746481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8072b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01f8072320 tx=0x7f01ec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d65ee22c886fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.302+0000 7f01f7e00640 1 -- 172.21.15.43:0/858746481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec036070 con 0x7f01f81467c0 2024-04-26T20:25:53.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.302+0000 7f01f7e00640 1 -- 172.21.15.43:0/858746481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec031600 con 0x7f01f81467c0 2024-04-26T20:25:53.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.302+0000 7f01f7e00640 1 -- 172.21.15.43:0/858746481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0319d0 con 0x7f01f81467c0 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 -- 172.21.15.43:0/858746481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 msgr2=0x7f01f8072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 --2- 172.21.15.43:0/858746481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8072b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f01f8072320 tx=0x7f01ec02a730 comp rx=0 tx=0).stop 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 -- 172.21.15.43:0/858746481 shutdown_connections 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 --2- 172.21.15.43:0/858746481 >> v2:172.21.15.164:3300/0 conn(0x7f01f80b3ce0 0x7f01f80d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 --2- 172.21.15.43:0/858746481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8072b70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 -- 172.21.15.43:0/858746481 >> 172.21.15.43:0/858746481 conn(0x7f01f80b2900 msgr2=0x7f01f80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 -- 172.21.15.43:0/858746481 shutdown_connections 2024-04-26T20:25:53.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.303+0000 7f0200200640 1 -- 172.21.15.43:0/858746481 wait complete. 2024-04-26T20:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.305+0000 7f0200200640 1 Processor -- start 2024-04-26T20:25:53.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.305+0000 7f0200200640 1 -- start start 2024-04-26T20:25:53.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f80b3ce0 0x7f01f81431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01f815c800 0x7f01f81f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f806f000 con 0x7f01f81467c0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8073a60 con 0x7f01f80b3ce0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f0200200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01f806f180 con 0x7f01f815c800 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f01fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f01fce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53364/0 (socket says 172.21.15.43:53364) 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.306+0000 7f01fce00640 1 -- 172.21.15.43:0/1464217425 learned_addr learned my addr 172.21.15.43:0/1464217425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fd800640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f80b3ce0 0x7f01f81431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fe200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01f815c800 0x7f01f81f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 -- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01f815c800 msgr2=0x7f01f81f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01f815c800 0x7f01f81f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 -- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f80b3ce0 msgr2=0x7f01f81431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f80b3ce0 0x7f01f81431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 -- 172.21.15.43:0/1464217425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81f8ff0 con 0x7f01f81467c0 2024-04-26T20:25:53.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.307+0000 7f01fce00640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01e400e810 tx=0x7f01e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.308+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400f040 con 0x7f01f81467c0 2024-04-26T20:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.308+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e4004030 con 0x7f01f81467c0 2024-04-26T20:25:53.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.308+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4004400 con 0x7f01f81467c0 2024-04-26T20:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.308+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81f9290 con 0x7f01f81467c0 2024-04-26T20:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.308+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f813d240 con 0x7f01f81467c0 2024-04-26T20:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.309+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f8069350 con 0x7f01f81467c0 2024-04-26T20:25:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.310+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01e400c6d0 con 0x7f01f81467c0 2024-04-26T20:25:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.311+0000 7f01f5600640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.311+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f01e408b520 con 0x7f01f81467c0 2024-04-26T20:25:53.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.311+0000 7f01fd800640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.312+0000 7f01fd800640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01ec033000 tx=0x7f01ec009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.315+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01e4017020 con 0x7f01f81467c0 2024-04-26T20:25:53.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.529+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01f80b1530 con 0x7f01f81467c0 2024-04-26T20:25:53.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.531+0000 7f01f5600640 1 -- 172.21.15.43:0/1464217425 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12860 (secure 0 0 0) 0x7f01e4059640 con 0x7f01f81467c0 2024-04-26T20:25:53.532 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:53.532 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:25:53.238097+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:25:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 msgr2=0x7f01d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01ec033000 tx=0x7f01ec009000 comp rx=0 tx=0).stop 2024-04-26T20:25:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 msgr2=0x7f01f8143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01e400e810 tx=0x7f01e400ecd0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 shutdown_connections 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01f815c800 0x7f01f81f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f81467c0 0x7f01f8143cd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 --2- 172.21.15.43:0/1464217425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01f80b3ce0 0x7f01f81431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 >> 172.21.15.43:0/1464217425 conn(0x7f01f80b2900 msgr2=0x7f01f806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 shutdown_connections 2024-04-26T20:25:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.534+0000 7f0200200640 1 -- 172.21.15.43:0/1464217425 wait complete. 2024-04-26T20:25:53.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:25:53.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:25:53.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.773+0000 7f4481200640 1 Processor -- start 2024-04-26T20:25:53.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.774+0000 7f4481200640 1 -- start start 2024-04-26T20:25:53.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.775+0000 7f4481200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 0x7f447c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.775+0000 7f4481200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f447c15d9c0 0x7f447c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.775+0000 7f4481200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0b3900 con 0x7f447c0b5410 2024-04-26T20:25:53.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.775+0000 7f4481200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f447c15abc0 con 0x7f447c15d9c0 2024-04-26T20:25:53.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.775+0000 7f4481200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f447c0b3b70 con 0x7f447c15a7e0 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447b400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 0x7f447c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447b400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 0x7f447c0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:33216/0 (socket says 172.21.15.43:33216) 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f4473e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f447c0b5410 0x7f447c15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:51056/0 (socket says 172.21.15.43:51056) 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447b400640 1 -- 172.21.15.43:0/2683301610 learned_addr learned my addr 172.21.15.43:0/2683301610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447be00640 1 --2- 172.21.15.43:0/2683301610 >> v2:172.21.15.43:3300/0 conn(0x7f447c15d9c0 0x7f447c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447be00640 1 -- 172.21.15.43:0/2683301610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 msgr2=0x7f447c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447be00640 1 --2- 172.21.15.43:0/2683301610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 0x7f447c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447be00640 1 -- 172.21.15.43:0/2683301610 >> v1:172.21.15.164:6789/0 conn(0x7f447c0b5410 legacy=0x7f447c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.776+0000 7f447be00640 1 -- 172.21.15.43:0/2683301610 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0b3e50 con 0x7f447c15d9c0 2024-04-26T20:25:53.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.778+0000 7f447be00640 1 --2- 172.21.15.43:0/2683301610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15d9c0 0x7f447c0b5890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4474009d90 tx=0x7f447402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa569824ace6bc9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.778+0000 7f447aa00640 1 -- 172.21.15.43:0/2683301610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474007740 con 0x7f447c15d9c0 2024-04-26T20:25:53.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.778+0000 7f447aa00640 1 -- 172.21.15.43:0/2683301610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44740078e0 con 0x7f447c15d9c0 2024-04-26T20:25:53.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.778+0000 7f447aa00640 1 -- 172.21.15.43:0/2683301610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474007cb0 con 0x7f447c15d9c0 2024-04-26T20:25:53.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 -- 172.21.15.43:0/2683301610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15d9c0 msgr2=0x7f447c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 --2- 172.21.15.43:0/2683301610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15d9c0 0x7f447c0b5890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4474009d90 tx=0x7f447402a660 comp rx=0 tx=0).stop 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 -- 172.21.15.43:0/2683301610 shutdown_connections 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 --2- 172.21.15.43:0/2683301610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15d9c0 0x7f447c0b5890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 --2- 172.21.15.43:0/2683301610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15a7e0 0x7f447c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 -- 172.21.15.43:0/2683301610 >> 172.21.15.43:0/2683301610 conn(0x7f447c068090 msgr2=0x7f447c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 -- 172.21.15.43:0/2683301610 shutdown_connections 2024-04-26T20:25:53.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.779+0000 7f4481200640 1 -- 172.21.15.43:0/2683301610 wait complete. 2024-04-26T20:25:53.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.781+0000 7f4481200640 1 Processor -- start 2024-04-26T20:25:53.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.781+0000 7f4481200640 1 -- start start 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c0b5410 0x7f447c14d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15d9c0 0x7f447c1595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f447c0b4b90 con 0x7f447c15a7e0 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f447c15e010 con 0x7f447c0b5410 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4481200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f447c15e2e0 con 0x7f447c15d9c0 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53370/0 (socket says 172.21.15.43:53370) 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 -- 172.21.15.43:0/655034471 learned_addr learned my addr 172.21.15.43:0/655034471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f447be00640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15d9c0 0x7f447c1595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 -- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15d9c0 msgr2=0x7f447c1595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15d9c0 0x7f447c1595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 -- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c0b5410 msgr2=0x7f447c14d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.782+0000 7f4473e00640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c0b5410 0x7f447c14d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4473e00640 1 -- 172.21.15.43:0/655034471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1e65c0 con 0x7f447c15a7e0 2024-04-26T20:25:53.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4473e00640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f446400c400 tx=0x7f446400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44640074e0 con 0x7f447c15a7e0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4464007680 con 0x7f447c15a7e0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1e68a0 con 0x7f447c15a7e0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464004a00 con 0x7f447c15a7e0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.783+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1e6e60 con 0x7f447c15a7e0 2024-04-26T20:25:53.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.784+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c0506c0 con 0x7f447c15a7e0 2024-04-26T20:25:53.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.785+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f446401a020 con 0x7f447c15a7e0 2024-04-26T20:25:53.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.786+0000 7f4473400640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 0x7f445806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:53.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.787+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f446408b100 con 0x7f447c15a7e0 2024-04-26T20:25:53.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.787+0000 7f447b400640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 0x7f445806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:53.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.787+0000 7f447b400640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 0x7f445806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f446c0094a0 tx=0x7f446c0093f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:53.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:53.791+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4464010300 con 0x7f447c15a7e0 2024-04-26T20:25:54.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.004+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f447c0af8c0 con 0x7f447c15a7e0 2024-04-26T20:25:54.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.004+0000 7f4473400640 1 -- 172.21.15.43:0/655034471 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f44640597e0 con 0x7f447c15a7e0 2024-04-26T20:25:54.005 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:25:54.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 msgr2=0x7f445806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 0x7f445806edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f446c0094a0 tx=0x7f446c0093f0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 msgr2=0x7f447c1e5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f446400c400 tx=0x7f446400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 shutdown_connections 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f445806c8f0 0x7f445806edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f447c15d9c0 0x7f447c1595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f447c15a7e0 0x7f447c1e5b00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 --2- 172.21.15.43:0/655034471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f447c0b5410 0x7f447c14d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.007+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 >> 172.21.15.43:0/655034471 conn(0x7f447c068090 msgr2=0x7f447c15baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.008+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 shutdown_connections 2024-04-26T20:25:54.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.008+0000 7f4481200640 1 -- 172.21.15.43:0/655034471 wait complete. 2024-04-26T20:25:54.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:25:54.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.234+0000 7f4826800640 1 Processor -- start 2024-04-26T20:25:54.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.234+0000 7f4826800640 1 -- start start 2024-04-26T20:25:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f482015b900 0x7f48200b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f482015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f482015fce0 0x7f48200724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4820160130 con 0x7f482015fce0 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4820071d20 con 0x7f482015b900 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4826800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f48200b4f00 con 0x7f482006f840 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4825e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f482015fce0 0x7f48200724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4825e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f482015fce0 0x7f48200724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53384/0 (socket says 172.21.15.43:53384) 2024-04-26T20:25:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.236+0000 7f4825e00640 1 -- 172.21.15.43:0/710518267 learned_addr learned my addr 172.21.15.43:0/710518267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825400640 1 --2- 172.21.15.43:0/710518267 >> v2:172.21.15.164:3300/0 conn(0x7f482015b900 0x7f48200b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4824a00640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f482015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825e00640 1 -- 172.21.15.43:0/710518267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 msgr2=0x7f482015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825e00640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f482015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825e00640 1 -- 172.21.15.43:0/710518267 >> v2:172.21.15.164:3300/0 conn(0x7f482015b900 msgr2=0x7f48200b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825e00640 1 --2- 172.21.15.43:0/710518267 >> v2:172.21.15.164:3300/0 conn(0x7f482015b900 0x7f48200b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4825e00640 1 -- 172.21.15.43:0/710518267 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200b51e0 con 0x7f482015fce0 2024-04-26T20:25:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.237+0000 7f4824a00640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f482015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.238+0000 7f4825e00640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015fce0 0x7f48200724b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48200b32c0 tx=0x7f481802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edd9a1151e7ec0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.238+0000 7f481fe00640 1 -- 172.21.15.43:0/710518267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818036070 con 0x7f482015fce0 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f481fe00640 1 -- 172.21.15.43:0/710518267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4818031600 con 0x7f482015fce0 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f481fe00640 1 -- 172.21.15.43:0/710518267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180319d0 con 0x7f482015fce0 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 -- 172.21.15.43:0/710518267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015fce0 msgr2=0x7f48200724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015fce0 0x7f48200724b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48200b32c0 tx=0x7f481802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 -- 172.21.15.43:0/710518267 shutdown_connections 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015fce0 0x7f48200724b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 --2- 172.21.15.43:0/710518267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f482015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 --2- 172.21.15.43:0/710518267 >> v2:172.21.15.164:3300/0 conn(0x7f482015b900 0x7f48200b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 -- 172.21.15.43:0/710518267 >> 172.21.15.43:0/710518267 conn(0x7f48200b26b0 msgr2=0x7f48200b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 -- 172.21.15.43:0/710518267 shutdown_connections 2024-04-26T20:25:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.239+0000 7f4826800640 1 -- 172.21.15.43:0/710518267 wait complete. 2024-04-26T20:25:54.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.241+0000 7f4826800640 1 Processor -- start 2024-04-26T20:25:54.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.241+0000 7f4826800640 1 -- start start 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f4820073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f482015fce0 0x7f48200ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48200b54c0 con 0x7f482015b900 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48200b4c10 con 0x7f482015fce0 2024-04-26T20:25:54.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.242+0000 7f4826800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f48200b5640 con 0x7f482006f840 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4825400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f4820073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53392/0 (socket says 172.21.15.43:53392) 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 -- 172.21.15.43:0/1086494490 learned_addr learned my addr 172.21.15.43:0/1086494490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4825e00640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f482015fce0 0x7f48200ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 -- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 msgr2=0x7f4820073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f4820073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 -- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f482015fce0 msgr2=0x7f48200ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f482015fce0 0x7f48200ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.243+0000 7f4824a00640 1 -- 172.21.15.43:0/1086494490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48200ab660 con 0x7f482015b900 2024-04-26T20:25:54.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f4824a00640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c7c0 tx=0x7f480c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c015070 con 0x7f482015b900 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480c007a30 con 0x7f482015b900 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c007e00 con 0x7f482015b900 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48200caad0 con 0x7f482015b900 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.244+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48200cb090 con 0x7f482015b900 2024-04-26T20:25:54.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.245+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820069350 con 0x7f482015b900 2024-04-26T20:25:54.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.247+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f480c018070 con 0x7f482015b900 2024-04-26T20:25:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.247+0000 7f481d600640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 0x7f480006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.248+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f480c08b5b0 con 0x7f482015b900 2024-04-26T20:25:54.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.248+0000 7f4825400640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 0x7f480006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.248+0000 7f4825400640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 0x7f480006ee00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4814006fd0 tx=0x7f4814008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.253+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f480c059ea0 con 0x7f482015b900 2024-04-26T20:25:54.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.460+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4820158b90 con 0x7f482015b900 2024-04-26T20:25:54.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.461+0000 7f481d600640 1 -- 172.21.15.43:0/1086494490 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f480c0597a0 con 0x7f482015b900 2024-04-26T20:25:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:25:54.462 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-26T20:25:54.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 msgr2=0x7f480006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 0x7f480006ee00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4814006fd0 tx=0x7f4814008040 comp rx=0 tx=0).stop 2024-04-26T20:25:54.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 msgr2=0x7f48200c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c7c0 tx=0x7f480c00cc80 comp rx=0 tx=0).stop 2024-04-26T20:25:54.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 shutdown_connections 2024-04-26T20:25:54.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f480006c940 0x7f480006ee00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f482015fce0 0x7f48200ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f482015b900 0x7f48200c9da0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 --2- 172.21.15.43:0/1086494490 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f482006f840 0x7f4820073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 >> 172.21.15.43:0/1086494490 conn(0x7f48200b26b0 msgr2=0x7f482015ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:54.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 shutdown_connections 2024-04-26T20:25:54.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.464+0000 7f4826800640 1 -- 172.21.15.43:0/1086494490 wait complete. 2024-04-26T20:25:54.483 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:25:54.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:25:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.701+0000 7fed11000640 1 Processor -- start 2024-04-26T20:25:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.701+0000 7fed11000640 1 -- start start 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed11000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c070510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed11000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed0c0b6b40 0x7fed0c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed11000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0c0b1480 con 0x7fed0c073940 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed11000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0d2780 con 0x7fed0c0b6b40 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed11000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c073e00 con 0x7fed0c0b10a0 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed0aa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed0c0b6b40 0x7fed0c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed0be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed0c073940 0x7fed0c0d2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41752/0 (socket says 172.21.15.43:41752) 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed0be00640 1 -- 172.21.15.43:0/3785415731 learned_addr learned my addr 172.21.15.43:0/3785415731 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.703+0000 7fed0b400640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0aa00640 1 -- 172.21.15.43:0/3785415731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 msgr2=0x7fed0c070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0aa00640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0aa00640 1 -- 172.21.15.43:0/3785415731 >> v1:172.21.15.43:6789/0 conn(0x7fed0c073940 legacy=0x7fed0c0d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0aa00640 1 -- 172.21.15.43:0/3785415731 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0b4c40 con 0x7fed0c0b6b40 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0a000640 1 -- 172.21.15.43:0/3785415731 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221660221 0 0) 0x7fed0c0b1480 con 0x7fed0c073940 2024-04-26T20:25:54.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.704+0000 7fed0b400640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.705+0000 7fed0aa00640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c0b6b40 0x7fed0c06ebf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fecf80097a0 tx=0x7fecf802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ca9658257f225f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.705+0000 7fed0a000640 1 -- 172.21.15.43:0/3785415731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8037070 con 0x7fed0c0b6b40 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.705+0000 7fed0a000640 1 -- 172.21.15.43:0/3785415731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf802adb0 con 0x7fed0c0b6b40 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.705+0000 7fed0a000640 1 -- 172.21.15.43:0/3785415731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8032660 con 0x7fed0c0b6b40 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 -- 172.21.15.43:0/3785415731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c0b6b40 msgr2=0x7fed0c06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c0b6b40 0x7fed0c06ebf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fecf80097a0 tx=0x7fecf802a060 comp rx=0 tx=0).stop 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 -- 172.21.15.43:0/3785415731 shutdown_connections 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c0b6b40 0x7fed0c06ebf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 --2- 172.21.15.43:0/3785415731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 -- 172.21.15.43:0/3785415731 >> 172.21.15.43:0/3785415731 conn(0x7fed0c068090 msgr2=0x7fed0c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 -- 172.21.15.43:0/3785415731 shutdown_connections 2024-04-26T20:25:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.706+0000 7fed11000640 1 -- 172.21.15.43:0/3785415731 wait complete. 2024-04-26T20:25:54.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.708+0000 7fed11000640 1 Processor -- start 2024-04-26T20:25:54.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.708+0000 7fed11000640 1 -- start start 2024-04-26T20:25:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed0c144620 0x7fed0c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c0b65b0 con 0x7fed0c073940 2024-04-26T20:25:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c070a50 con 0x7fed0c144620 2024-04-26T20:25:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed11000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c06f130 con 0x7fed0c0b10a0 2024-04-26T20:25:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53410/0 (socket says 172.21.15.43:53410) 2024-04-26T20:25:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 -- 172.21.15.43:0/2141338328 learned_addr learned my addr 172.21.15.43:0/2141338328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0aa00640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0be00640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed0c144620 0x7fed0c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 -- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 msgr2=0x7fed0c160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 -- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed0c144620 msgr2=0x7fed0c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed0c144620 0x7fed0c15f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0b400640 1 -- 172.21.15.43:0/2141338328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c161230 con 0x7fed0c073940 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.709+0000 7fed0aa00640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed0b400640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009fc0 tx=0x7fecfc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01a070 con 0x7fed0c073940 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc003b90 con 0x7fed0c073940 2024-04-26T20:25:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004030 con 0x7fed0c073940 2024-04-26T20:25:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c1614d0 con 0x7fed0c073940 2024-04-26T20:25:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.710+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c161a90 con 0x7fed0c073940 2024-04-26T20:25:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.711+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c069a60 con 0x7fed0c073940 2024-04-26T20:25:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.712+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecfc01d070 con 0x7fed0c073940 2024-04-26T20:25:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.713+0000 7fed03400640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.713+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fecfc08ba80 con 0x7fed0c073940 2024-04-26T20:25:54.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.714+0000 7fed0aa00640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:54.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.714+0000 7fed0aa00640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 0x7fecec06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecf8034000 tx=0x7fecf802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:54.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.718+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecfc08de20 con 0x7fed0c073940 2024-04-26T20:25:54.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:54.929+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fed0c0734f0 con 0x7fed0c073940 2024-04-26T20:25:55.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.934+0000 7fed03400640 1 -- 172.21.15.43:0/2141338328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fecfc010070 con 0x7fed0c073940 2024-04-26T20:25:55.935 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-26T20:25:55.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 msgr2=0x7fecec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:55.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 0x7fecec06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecf8034000 tx=0x7fecf802e000 comp rx=0 tx=0).stop 2024-04-26T20:25:55.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 msgr2=0x7fed0c13fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:55.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009fc0 tx=0x7fecfc00d560 comp rx=0 tx=0).stop 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 shutdown_connections 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fecec06c8f0 0x7fecec06edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fed0c0b10a0 0x7fed0c160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fed0c144620 0x7fed0c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 --2- 172.21.15.43:0/2141338328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed0c073940 0x7fed0c13fde0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 >> 172.21.15.43:0/2141338328 conn(0x7fed0c068090 msgr2=0x7fed0c0d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.938+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 shutdown_connections 2024-04-26T20:25:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:55.939+0000 7fed11000640 1 -- 172.21.15.43:0/2141338328 wait complete. 2024-04-26T20:25:55.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T20:25:56.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.169+0000 7ff677000640 1 Processor -- start 2024-04-26T20:25:56.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.169+0000 7ff677000640 1 -- start start 2024-04-26T20:25:56.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff677000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6700c9380 0x7ff67006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff677000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff67006e4f0 0x7ff67015e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff677000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6700d1fe0 con 0x7ff6700d1a40 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff677000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff67006dc40 con 0x7ff6700c9380 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff677000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff6700d3b70 con 0x7ff67006e4f0 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff66fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6700c9380 0x7ff67006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff66f400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6700d1a40 0x7ff67006dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51076/0 (socket says 172.21.15.43:51076) 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff66f400640 1 -- 172.21.15.43:0/2100399046 learned_addr learned my addr 172.21.15.43:0/2100399046 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.171+0000 7ff675000640 1 --2- 172.21.15.43:0/2100399046 >> v2:172.21.15.164:3300/0 conn(0x7ff67006e4f0 0x7ff67015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.172+0000 7ff66fe00640 1 -- 172.21.15.43:0/2100399046 >> v2:172.21.15.164:3300/0 conn(0x7ff67006e4f0 msgr2=0x7ff67015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.172+0000 7ff66fe00640 1 --2- 172.21.15.43:0/2100399046 >> v2:172.21.15.164:3300/0 conn(0x7ff67006e4f0 0x7ff67015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.172+0000 7ff66fe00640 1 -- 172.21.15.43:0/2100399046 >> v1:172.21.15.43:6789/0 conn(0x7ff6700d1a40 legacy=0x7ff67006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.172+0000 7ff66fe00640 1 -- 172.21.15.43:0/2100399046 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700d22c0 con 0x7ff6700c9380 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.172+0000 7ff66ea00640 1 -- 172.21.15.43:0/2100399046 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492315551 0 0) 0x7ff6700d1fe0 con 0x7ff6700d1a40 2024-04-26T20:25:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.173+0000 7ff66fe00640 1 --2- 172.21.15.43:0/2100399046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700c9380 0x7ff67006d700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff670073620 tx=0x7ff66002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b2837cf4e4a8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.173+0000 7ff66ea00640 1 -- 172.21.15.43:0/2100399046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037070 con 0x7ff6700c9380 2024-04-26T20:25:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.173+0000 7ff66ea00640 1 -- 172.21.15.43:0/2100399046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff660032440 con 0x7ff6700c9380 2024-04-26T20:25:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.173+0000 7ff66ea00640 1 -- 172.21.15.43:0/2100399046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660032810 con 0x7ff6700c9380 2024-04-26T20:25:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 -- 172.21.15.43:0/2100399046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700c9380 msgr2=0x7ff67006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 --2- 172.21.15.43:0/2100399046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700c9380 0x7ff67006d700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff670073620 tx=0x7ff66002a250 comp rx=0 tx=0).stop 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 -- 172.21.15.43:0/2100399046 shutdown_connections 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 --2- 172.21.15.43:0/2100399046 >> v2:172.21.15.164:3300/0 conn(0x7ff67006e4f0 0x7ff67015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 --2- 172.21.15.43:0/2100399046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700c9380 0x7ff67006d700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 -- 172.21.15.43:0/2100399046 >> 172.21.15.43:0/2100399046 conn(0x7ff6700d2d20 msgr2=0x7ff6700d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 -- 172.21.15.43:0/2100399046 shutdown_connections 2024-04-26T20:25:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.174+0000 7ff677000640 1 -- 172.21.15.43:0/2100399046 wait complete. 2024-04-26T20:25:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.176+0000 7ff677000640 1 Processor -- start 2024-04-26T20:25:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.176+0000 7ff677000640 1 -- start start 2024-04-26T20:25:56.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6700c9380 0x7ff6701f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6700d1a40 0x7ff67015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700cd9d0 con 0x7ff67006e4f0 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700cd6f0 con 0x7ff6700c9380 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff677000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff67015eed0 con 0x7ff6700d1a40 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff66fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff66fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55296/0 (socket says 172.21.15.43:55296) 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.177+0000 7ff66fe00640 1 -- 172.21.15.43:0/1593077284 learned_addr learned my addr 172.21.15.43:0/1593077284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66f400640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6700c9380 0x7ff6701f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 -- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6700d1a40 msgr2=0x7ff67015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6700d1a40 0x7ff67015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 -- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6700c9380 msgr2=0x7ff6701f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6700c9380 0x7ff6701f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 -- 172.21.15.43:0/1593077284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700b5b90 con 0x7ff67006e4f0 2024-04-26T20:25:56.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff66fe00640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff660009860 tx=0x7ff6600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.178+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660040020 con 0x7ff67006e4f0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.179+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff660038040 con 0x7ff67006e4f0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.179+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037070 con 0x7ff67006e4f0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.179+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6700b5e70 con 0x7ff67006e4f0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.179+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701f6170 con 0x7ff67006e4f0 2024-04-26T20:25:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.180+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff670069350 con 0x7ff67006e4f0 2024-04-26T20:25:56.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.181+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff660043020 con 0x7ff67006e4f0 2024-04-26T20:25:56.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.182+0000 7ff667e00640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 0x7ff65006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:56.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.182+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7ff6600ac270 con 0x7ff67006e4f0 2024-04-26T20:25:56.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.182+0000 7ff66f400640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 0x7ff65006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:56.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.183+0000 7ff66f400640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 0x7ff65006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6700d34d0 tx=0x7ff65c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.187+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff66004a020 con 0x7ff67006e4f0 2024-04-26T20:25:56.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:56.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:25:56.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.391+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff6700d37c0 con 0x7ff67006e4f0 2024-04-26T20:25:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.941+0000 7ff667e00640 1 -- 172.21.15.43:0/1593077284 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7ff660032e20 con 0x7ff67006e4f0 2024-04-26T20:25:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-26T20:25:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.944+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 msgr2=0x7ff65006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.944+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 0x7ff65006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6700d34d0 tx=0x7ff65c009040 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 msgr2=0x7ff6701617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff660009860 tx=0x7ff6600046c0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 shutdown_connections 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff65006c8f0 0x7ff65006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6700d1a40 0x7ff67015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6700c9380 0x7ff6701f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 --2- 172.21.15.43:0/1593077284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff67006e4f0 0x7ff6701617a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 >> 172.21.15.43:0/1593077284 conn(0x7ff6700d2d20 msgr2=0x7ff6700c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 shutdown_connections 2024-04-26T20:25:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:56.945+0000 7ff677000640 1 -- 172.21.15.43:0/1593077284 wait complete. 2024-04-26T20:25:56.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:25:57.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.212+0000 7f6eb8400640 1 Processor -- start 2024-04-26T20:25:57.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.212+0000 7f6eb8400640 1 -- start start 2024-04-26T20:25:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb8400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6eb00d2d70 0x7f6eb00cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb8400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6eb00d31d0 0x7f6eb00d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb8400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb00b5160 con 0x7f6eb00d3780 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb8400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00b5d70 con 0x7f6eb00d2d70 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb8400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00b5440 con 0x7f6eb00d31d0 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb5a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6eb00d3780 0x7f6eb00d3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51088/0 (socket says 172.21.15.43:51088) 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb5a00640 1 -- 172.21.15.43:0/3661627488 learned_addr learned my addr 172.21.15.43:0/3661627488 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3661627488 >> v2:172.21.15.43:3300/0 conn(0x7f6eb00d2d70 0x7f6eb00cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.214+0000 7f6eb6400640 1 --2- 172.21.15.43:0/3661627488 >> v2:172.21.15.164:3300/0 conn(0x7f6eb00d31d0 0x7f6eb00d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.215+0000 7f6eb5000640 1 -- 172.21.15.43:0/3661627488 >> v2:172.21.15.164:3300/0 conn(0x7f6eb00d31d0 msgr2=0x7f6eb00d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.215+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3661627488 >> v2:172.21.15.164:3300/0 conn(0x7f6eb00d31d0 0x7f6eb00d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.215+0000 7f6eb5000640 1 -- 172.21.15.43:0/3661627488 >> v1:172.21.15.43:6789/0 conn(0x7f6eb00d3780 legacy=0x7f6eb00d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.215+0000 7f6eb5000640 1 -- 172.21.15.43:0/3661627488 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00b5720 con 0x7f6eb00d2d70 2024-04-26T20:25:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.216+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3661627488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb00cfd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6eb008e6a0 tx=0x7f6e9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23bdeffe86ede23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:57.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.216+0000 7f6eafe00640 1 -- 172.21.15.43:0/3661627488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6eb00d2d70 2024-04-26T20:25:57.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.216+0000 7f6eafe00640 1 -- 172.21.15.43:0/3661627488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c032440 con 0x7f6eb00d2d70 2024-04-26T20:25:57.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.216+0000 7f6eafe00640 1 -- 172.21.15.43:0/3661627488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c032810 con 0x7f6eb00d2d70 2024-04-26T20:25:57.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 -- 172.21.15.43:0/3661627488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 msgr2=0x7f6eb00cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3661627488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb00cfd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6eb008e6a0 tx=0x7f6e9c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 -- 172.21.15.43:0/3661627488 shutdown_connections 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3661627488 >> v2:172.21.15.164:3300/0 conn(0x7f6eb00d31d0 0x7f6eb00d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3661627488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb00cfd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 -- 172.21.15.43:0/3661627488 >> 172.21.15.43:0/3661627488 conn(0x7f6eb008ec50 msgr2=0x7f6eb009e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 -- 172.21.15.43:0/3661627488 shutdown_connections 2024-04-26T20:25:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.217+0000 7f6eb8400640 1 -- 172.21.15.43:0/3661627488 wait complete. 2024-04-26T20:25:57.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.219+0000 7f6eb8400640 1 Processor -- start 2024-04-26T20:25:57.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.219+0000 7f6eb8400640 1 -- start start 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6eb00d31d0 0x7f6eb014f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6eb00d3780 0x7f6eb00ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00b4920 con 0x7f6eb00d2d70 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb015f5e0 con 0x7f6eb00d31d0 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb8400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b4aa0 con 0x7f6eb00d3780 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55310/0 (socket says 172.21.15.43:55310) 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb5a00640 1 -- 172.21.15.43:0/3733926472 learned_addr learned my addr 172.21.15.43:0/3733926472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6eb00d31d0 0x7f6eb014f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.220+0000 7f6eb6400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6eb00d3780 0x7f6eb00ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 -- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6eb00d3780 msgr2=0x7f6eb00ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6eb00d3780 0x7f6eb00ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 -- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6eb00d31d0 msgr2=0x7f6eb014f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6eb00d31d0 0x7f6eb014f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 -- 172.21.15.43:0/3733926472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00ceb90 con 0x7f6eb00d2d70 2024-04-26T20:25:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb5a00640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c3c0 tx=0x7f6ea400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4017070 con 0x7f6eb00d2d70 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea4005ce0 con 0x7f6eb00d2d70 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40072f0 con 0x7f6eb00d2d70 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb00cee70 con 0x7f6eb00d2d70 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.221+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb00cf430 con 0x7f6eb00d2d70 2024-04-26T20:25:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.222+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb00b4ea0 con 0x7f6eb00d2d70 2024-04-26T20:25:57.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.224+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ea401a070 con 0x7f6eb00d2d70 2024-04-26T20:25:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.225+0000 7f6ead600640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.225+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f6ea408aea0 con 0x7f6eb00d2d70 2024-04-26T20:25:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.225+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.226+0000 7f6eb5000640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 0x7f6e9006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.230+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6eb00b4ea0 con 0x7f6eb00d2d70 2024-04-26T20:25:57.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.478+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6eb00bae00 con 0x7f6eb00d2d70 2024-04-26T20:25:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.962+0000 7f6ead600640 1 -- 172.21.15.43:0/3733926472 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f6ea4059320 con 0x7f6eb00d2d70 2024-04-26T20:25:57.963 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.965+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 msgr2=0x7f6e9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.965+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 0x7f6e9006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.965+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 msgr2=0x7f6eb01f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.965+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c3c0 tx=0x7f6ea400c880 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 shutdown_connections 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e9006c8f0 0x7f6e9006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6eb00d3780 0x7f6eb00ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6eb00d31d0 0x7f6eb014f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 --2- 172.21.15.43:0/3733926472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb00d2d70 0x7f6eb01f62b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 >> 172.21.15.43:0/3733926472 conn(0x7f6eb008ec50 msgr2=0x7f6eb006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 shutdown_connections 2024-04-26T20:25:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:57.966+0000 7f6eb8400640 1 -- 172.21.15.43:0/3733926472 wait complete. 2024-04-26T20:25:58.016 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T20:25:58.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T20:25:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.195+0000 7f77cb400640 1 Processor -- start 2024-04-26T20:25:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.195+0000 7f77cb400640 1 -- start start 2024-04-26T20:25:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77cb400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77c40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77cb400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f77c406fec0 0x7f77c406ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77cb400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c406df30 con 0x7f77c406f230 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77cb400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f77c4146be0 con 0x7f77c406fec0 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77cb400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77c406e860 con 0x7f77c4146800 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77c9400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77c406f230 0x7f77c4071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51104/0 (socket says 172.21.15.43:51104) 2024-04-26T20:25:58.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.197+0000 7f77c9400640 1 -- 172.21.15.43:0/545009481 learned_addr learned my addr 172.21.15.43:0/545009481 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3e00640 1 --2- 172.21.15.43:0/545009481 >> v2:172.21.15.164:3300/0 conn(0x7f77c406fec0 0x7f77c406ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c8a00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77c40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109597647 0 0) 0x7f77c406df30 con 0x7f77c406f230 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77ac003520 con 0x7f77c406f230 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3902765273 0 0) 0x7f77ac003520 con 0x7f77c406f230 2024-04-26T20:25:58.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 msgr2=0x7f77c40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77c40cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 >> v2:172.21.15.164:3300/0 conn(0x7f77c406fec0 msgr2=0x7f77c406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 --2- 172.21.15.43:0/545009481 >> v2:172.21.15.164:3300/0 conn(0x7f77c406fec0 0x7f77c406ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c406e210 con 0x7f77c406f230 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f77bc0019f0 con 0x7f77c406f230 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 >> v1:172.21.15.43:6789/0 conn(0x7f77c406f230 legacy=0x7f77c4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77ac004800 0x7f77ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77c406e860 con 0x7f77ac0039b0 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac00b9f0 con 0x7f77ac004800 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.198+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77ac00bbf0 con 0x7f77c4146800 2024-04-26T20:25:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c9400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 -- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 msgr2=0x7f77ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c3e00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77ac004800 0x7f77ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 -- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77ac004800 msgr2=0x7f77ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77ac004800 0x7f77ac004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c8a00640 1 -- 172.21.15.43:0/545009481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac003520 con 0x7f77ac0039b0 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.199+0000 7f77c9400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c4146800 0x7f77ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.200+0000 7f77c8a00640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77ac003dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77b800ba00 tx=0x7f77ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0519dabfb54c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.200+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8030070 con 0x7f77ac0039b0 2024-04-26T20:25:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.200+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b802ad80 con 0x7f77ac0039b0 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.200+0000 7f77c3400640 1 -- 172.21.15.43:0/545009481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b800b600 con 0x7f77ac0039b0 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 -- 172.21.15.43:0/545009481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 msgr2=0x7f77ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 --2- 172.21.15.43:0/545009481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77ac003dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77b800ba00 tx=0x7f77ac005830 comp rx=0 tx=0).stop 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77c9400640 1 -- 172.21.15.43:0/545009481 reap_dead start 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 -- 172.21.15.43:0/545009481 shutdown_connections 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 -- 172.21.15.43:0/545009481 >> 172.21.15.43:0/545009481 conn(0x7f77c40b2790 msgr2=0x7f77c40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 -- 172.21.15.43:0/545009481 shutdown_connections 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.201+0000 7f77cb400640 1 -- 172.21.15.43:0/545009481 wait complete. 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.203+0000 7f77cb400640 1 Processor -- start 2024-04-26T20:25:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.203+0000 7f77cb400640 1 -- start start 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77c4141700 0x7f77c40b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c41411b0 0x7f77c40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4072570 con 0x7f77ac0039b0 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77c40d0520 con 0x7f77c4141700 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77cb400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77c40726f0 con 0x7f77c41411b0 2024-04-26T20:25:58.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77c8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77c8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55320/0 (socket says 172.21.15.43:55320) 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.204+0000 7f77c8a00640 1 -- 172.21.15.43:0/3677192578 learned_addr learned my addr 172.21.15.43:0/3677192578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c3e00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77c4141700 0x7f77c40b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c9400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c41411b0 0x7f77c40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 -- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c41411b0 msgr2=0x7f77c40b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c41411b0 0x7f77c40b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 -- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77c4141700 msgr2=0x7f77c40b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77c4141700 0x7f77c40b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 -- 172.21.15.43:0/3677192578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c4140fa0 con 0x7f77ac0039b0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c8a00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77b800bb20 tx=0x7f77b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8043020 con 0x7f77ac0039b0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b803f680 con 0x7f77ac0039b0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b800b600 con 0x7f77ac0039b0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.205+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c4142240 con 0x7f77ac0039b0 2024-04-26T20:25:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.206+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c4142800 con 0x7f77ac0039b0 2024-04-26T20:25:58.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.207+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c4069350 con 0x7f77ac0039b0 2024-04-26T20:25:58.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.208+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77b8046020 con 0x7f77ac0039b0 2024-04-26T20:25:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.209+0000 7f77c0c00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 0x7f77a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.209+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f77b80ac8c0 con 0x7f77ac0039b0 2024-04-26T20:25:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.209+0000 7f77c3e00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 0x7f77a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.210+0000 7f77c3e00640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 0x7f77a806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c41574d0 tx=0x7f77b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.214+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77b804d030 con 0x7f77ac0039b0 2024-04-26T20:25:58.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.428+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f77c415e860 con 0x7f77ac0039b0 2024-04-26T20:25:58.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.966+0000 7f77c0c00640 1 -- 172.21.15.43:0/3677192578 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f77b807ac50 con 0x7f77ac0039b0 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.969+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 msgr2=0x7f77a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.969+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 0x7f77a806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77c41574d0 tx=0x7f77b4008040 comp rx=0 tx=0).stop 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 msgr2=0x7f77c4140a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77b800bb20 tx=0x7f77b8004730 comp rx=0 tx=0).stop 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 shutdown_connections 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f77a806c8f0 0x7f77a806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77c41411b0 0x7f77c40b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77c4141700 0x7f77c40b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 --2- 172.21.15.43:0/3677192578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac0039b0 0x7f77c4140a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 >> 172.21.15.43:0/3677192578 conn(0x7f77c40b2790 msgr2=0x7f77c4071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 shutdown_connections 2024-04-26T20:25:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:58.970+0000 7f77cb400640 1 -- 172.21.15.43:0/3677192578 wait complete. 2024-04-26T20:25:58.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:25:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.210+0000 7fb9aac00640 1 Processor -- start 2024-04-26T20:25:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.210+0000 7fb9aac00640 1 -- start start 2024-04-26T20:25:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aac00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9a4072df0 0x7fb9a40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aac00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 0x7fb9a40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a40b0380 con 0x7fb9a415bd50 2024-04-26T20:25:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aac00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a415c130 con 0x7fb9a4072df0 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aac00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a406dc10 con 0x7fb9a40ccf80 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9a4072df0 0x7fb9a40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9a4072df0 0x7fb9a40b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55326/0 (socket says 172.21.15.43:55326) 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 -- 172.21.15.43:0/2223629894 learned_addr learned my addr 172.21.15.43:0/2223629894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9aa200640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 0x7fb9a40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 -- 172.21.15.43:0/2223629894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 msgr2=0x7fb9a40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 0x7fb9a40b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.212+0000 7fb9a8e00640 1 -- 172.21.15.43:0/2223629894 >> v1:172.21.15.164:6789/0 conn(0x7fb9a415bd50 legacy=0x7fb9a40726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.213+0000 7fb9a8e00640 1 -- 172.21.15.43:0/2223629894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a406de60 con 0x7fb9a4072df0 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.213+0000 7fb9aa200640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 0x7fb9a40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.214+0000 7fb9a8e00640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a4072df0 0x7fb9a40b4420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb990009870 tx=0x7fb99002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b579044c62312af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.214+0000 7fb9a3e00640 1 -- 172.21.15.43:0/2223629894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990037070 con 0x7fb9a4072df0 2024-04-26T20:25:59.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.214+0000 7fb9a3e00640 1 -- 172.21.15.43:0/2223629894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990032440 con 0x7fb9a4072df0 2024-04-26T20:25:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.214+0000 7fb9a3e00640 1 -- 172.21.15.43:0/2223629894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990032810 con 0x7fb9a4072df0 2024-04-26T20:25:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 -- 172.21.15.43:0/2223629894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a4072df0 msgr2=0x7fb9a40b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a4072df0 0x7fb9a40b4420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb990009870 tx=0x7fb99002a200 comp rx=0 tx=0).stop 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 -- 172.21.15.43:0/2223629894 shutdown_connections 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a40ccf80 0x7fb9a40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2223629894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a4072df0 0x7fb9a40b4420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 -- 172.21.15.43:0/2223629894 >> 172.21.15.43:0/2223629894 conn(0x7fb9a40b2640 msgr2=0x7fb9a40b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 -- 172.21.15.43:0/2223629894 shutdown_connections 2024-04-26T20:25:59.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.215+0000 7fb9aac00640 1 -- 172.21.15.43:0/2223629894 wait complete. 2024-04-26T20:25:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.217+0000 7fb9aac00640 1 Processor -- start 2024-04-26T20:25:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.217+0000 7fb9aac00640 1 -- start start 2024-04-26T20:25:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 0x7fb9a4149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 0x7fb9a4157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a40b3c20 con 0x7fb9a415bd50 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a40b3940 con 0x7fb9a40ccf80 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aac00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a40b3f00 con 0x7fb9a4072df0 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aa200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55338/0 (socket says 172.21.15.43:55338) 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9aa200640 1 -- 172.21.15.43:0/2119924862 learned_addr learned my addr 172.21.15.43:0/2119924862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9a8e00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 0x7fb9a4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.218+0000 7fb9a9800640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 0x7fb9a4149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 -- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 msgr2=0x7fb9a4149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 0x7fb9a4149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 -- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 msgr2=0x7fb9a4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:25:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 0x7fb9a4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 -- 172.21.15.43:0/2119924862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a4156160 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9a9800640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 0x7fb9a4149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9a8e00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 0x7fb9a4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aa200640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00cf20 tx=0x7fb99c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c01a070 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c00d840 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00dc30 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a4156440 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.219+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a4156a00 con 0x7fb9a415bd50 2024-04-26T20:25:59.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.220+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a40506c0 con 0x7fb9a415bd50 2024-04-26T20:25:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.222+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb99c01d070 con 0x7fb9a415bd50 2024-04-26T20:25:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.222+0000 7fb9a1600640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:25:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.223+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fb99c08c5b0 con 0x7fb9a415bd50 2024-04-26T20:25:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.223+0000 7fb9a9800640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:25:59.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.223+0000 7fb9a9800640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 0x7fb98406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb99800a830 tx=0x7fb998008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:25:59.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.227+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb99c05a8a0 con 0x7fb9a415bd50 2024-04-26T20:25:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:25:59.428+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb9a40b3360 con 0x7fb9a415bd50 2024-04-26T20:26:00.844 INFO:tasks.ceph.mds.b.smithi164.stderr:2024-04-26T20:26:00.843+0000 7f65d1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:26:01.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.024+0000 7fb9a1600640 1 -- 172.21.15.43:0/2119924862 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fb99c05a1a0 con 0x7fb9a415bd50 2024-04-26T20:26:01.025 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' created 2024-04-26T20:26:01.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.028+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 msgr2=0x7fb98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 0x7fb98406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb99800a830 tx=0x7fb998008040 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 msgr2=0x7fb9a40cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00cf20 tx=0x7fb99c00a7d0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 shutdown_connections 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a415bd50 0x7fb9a40cebd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9a40ccf80 0x7fb9a4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 --2- 172.21.15.43:0/2119924862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9a4072df0 0x7fb9a4149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 >> 172.21.15.43:0/2119924862 conn(0x7fb9a40b2640 msgr2=0x7fb9a406ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 shutdown_connections 2024-04-26T20:26:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.029+0000 7fb9aac00640 1 -- 172.21.15.43:0/2119924862 wait complete. 2024-04-26T20:26:01.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T20:26:01.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.243+0000 7fa93d000640 1 Processor -- start 2024-04-26T20:26:01.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.243+0000 7fa93d000640 1 -- start start 2024-04-26T20:26:01.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa93d000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa938145450 0x7fa9380b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:01.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa93d000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa938163690 con 0x7fa938163070 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa93d000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa93806ed50 con 0x7fa93806f830 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa93d000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa93806f030 con 0x7fa938145450 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa937400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa938145450 0x7fa9380b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa937e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa938163070 0x7fa938072c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51114/0 (socket says 172.21.15.43:51114) 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.245+0000 7fa937e00640 1 -- 172.21.15.43:0/496906274 learned_addr learned my addr 172.21.15.43:0/496906274 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:01.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.246+0000 7fa937400640 1 -- 172.21.15.43:0/496906274 >> v1:172.21.15.164:6789/0 conn(0x7fa93806f830 legacy=0x7fa938162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:01.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.246+0000 7fa937400640 1 -- 172.21.15.43:0/496906274 >> v1:172.21.15.43:6789/0 conn(0x7fa938163070 legacy=0x7fa938072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.246+0000 7fa937400640 1 -- 172.21.15.43:0/496906274 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380b5210 con 0x7fa938145450 2024-04-26T20:26:01.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.247+0000 7fa937400640 1 --2- 172.21.15.43:0/496906274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380b4cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9280097a0 tx=0x7fa92802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffc178adeeba3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:01.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.247+0000 7fa936000640 1 -- 172.21.15.43:0/496906274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928037070 con 0x7fa938145450 2024-04-26T20:26:01.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.247+0000 7fa936000640 1 -- 172.21.15.43:0/496906274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa928032440 con 0x7fa938145450 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.247+0000 7fa936000640 1 -- 172.21.15.43:0/496906274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928032810 con 0x7fa938145450 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 -- 172.21.15.43:0/496906274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 msgr2=0x7fa9380b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 --2- 172.21.15.43:0/496906274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380b4cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9280097a0 tx=0x7fa92802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 -- 172.21.15.43:0/496906274 shutdown_connections 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 --2- 172.21.15.43:0/496906274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380b4cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 -- 172.21.15.43:0/496906274 >> 172.21.15.43:0/496906274 conn(0x7fa9380b26b0 msgr2=0x7fa9380b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 -- 172.21.15.43:0/496906274 shutdown_connections 2024-04-26T20:26:01.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.248+0000 7fa93d000640 1 -- 172.21.15.43:0/496906274 wait complete. 2024-04-26T20:26:01.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.250+0000 7fa93d000640 1 Processor -- start 2024-04-26T20:26:01.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.250+0000 7fa93d000640 1 -- start start 2024-04-26T20:26:01.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa93806f830 0x7fa938142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:01.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:01.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa938163070 0x7fa9380c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:01.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380734d0 con 0x7fa938145450 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa938073350 con 0x7fa93806f830 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa93d000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa938073650 con 0x7fa938163070 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.251+0000 7fa936a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55354/0 (socket says 172.21.15.43:55354) 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 -- 172.21.15.43:0/1477530924 learned_addr learned my addr 172.21.15.43:0/1477530924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa937400640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa93806f830 0x7fa938142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa937e00640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa938163070 0x7fa9380c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 -- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa938163070 msgr2=0x7fa9380c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa938163070 0x7fa9380c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 -- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa93806f830 msgr2=0x7fa938142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa93806f830 0x7fa938142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 -- 172.21.15.43:0/1477530924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380c6630 con 0x7fa938145450 2024-04-26T20:26:01.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa936a00640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c400 tx=0x7fa92400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.252+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924011790 con 0x7fa938145450 2024-04-26T20:26:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.253+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924017070 con 0x7fa938145450 2024-04-26T20:26:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.253+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924011bb0 con 0x7fa938145450 2024-04-26T20:26:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.253+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9380c8660 con 0x7fa938145450 2024-04-26T20:26:01.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.253+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380c8c20 con 0x7fa938145450 2024-04-26T20:26:01.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.254+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa938069350 con 0x7fa938145450 2024-04-26T20:26:01.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.255+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa92401c460 con 0x7fa938145450 2024-04-26T20:26:01.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.256+0000 7fa92f400640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:01.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.256+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fa92408bdb0 con 0x7fa938145450 2024-04-26T20:26:01.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.256+0000 7fa937400640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:01.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.257+0000 7fa937400640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 0x7fa91806edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa92802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:01.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.261+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa924090070 con 0x7fa938145450 2024-04-26T20:26:01.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:01.465+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa9380c6860 con 0x7fa938145450 2024-04-26T20:26:02.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.038+0000 7fa92f400640 1 -- 172.21.15.43:0/1477530924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fa9380c6630 con 0x7fa938145450 2024-04-26T20:26:02.039 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.043+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 msgr2=0x7fa91806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.043+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 0x7fa91806edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa92802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.043+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 msgr2=0x7fa9380c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.043+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c400 tx=0x7fa92400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 shutdown_connections 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa91806c8f0 0x7fa91806edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa938163070 0x7fa9380c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa938145450 0x7fa9380c60f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 --2- 172.21.15.43:0/1477530924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa93806f830 0x7fa938142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.044+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 >> 172.21.15.43:0/1477530924 conn(0x7fa9380b26b0 msgr2=0x7fa938071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:02.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.046+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 shutdown_connections 2024-04-26T20:26:02.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.046+0000 7fa93d000640 1 -- 172.21.15.43:0/1477530924 wait complete. 2024-04-26T20:26:02.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T20:26:02.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.260+0000 7fc215c00640 1 Processor -- start 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.260+0000 7fc215c00640 1 -- start start 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.261+0000 7fc215c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 0x7fc2100b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.261+0000 7fc215c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc2100b4140 0x7fc21016c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.261+0000 7fc215c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2100b4550 con 0x7fc21016bea0 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.261+0000 7fc215c00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc21006e790 con 0x7fc2100b3460 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.261+0000 7fc215c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc21006ea70 con 0x7fc2100b4140 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.262+0000 7fc20ea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc21016bea0 0x7fc21006df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51126/0 (socket says 172.21.15.43:51126) 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.262+0000 7fc20ea00640 1 -- 172.21.15.43:0/2234744034 learned_addr learned my addr 172.21.15.43:0/2234744034 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.262+0000 7fc20e000640 1 --2- 172.21.15.43:0/2234744034 >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 0x7fc2100b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.262+0000 7fc20f400640 1 --2- 172.21.15.43:0/2234744034 >> v2:172.21.15.164:3300/0 conn(0x7fc2100b4140 0x7fc21016c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.262+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519136279 0 0) 0x7fc200001110 con 0x7fc21016bea0 2024-04-26T20:26:02.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.382+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f4003420 con 0x7fc21016bea0 2024-04-26T20:26:02.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1343771089 0 0) 0x7fc1f4003420 con 0x7fc21016bea0 2024-04-26T20:26:02.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20f400640 1 -- 172.21.15.43:0/2234744034 >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 msgr2=0x7fc2100b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:02.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20f400640 1 --2- 172.21.15.43:0/2234744034 >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 0x7fc2100b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20f400640 1 -- 172.21.15.43:0/2234744034 >> v1:172.21.15.43:6789/0 conn(0x7fc21016bea0 legacy=0x7fc21006df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20f400640 1 -- 172.21.15.43:0/2234744034 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21006ed50 con 0x7fc2100b4140 2024-04-26T20:26:02.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.383+0000 7fc20e000640 1 --2- 172.21.15.43:0/2234744034 >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 0x7fc2100b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:02.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.387+0000 7fc20f400640 1 --2- 172.21.15.43:0/2234744034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2100b4140 0x7fc21016c280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc210074370 tx=0x7fc20802a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed414994fd8a4992 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.387+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200001840 con 0x7fc2100b4140 2024-04-26T20:26:02.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.387+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc208007540 con 0x7fc2100b4140 2024-04-26T20:26:02.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.387+0000 7fc20d600640 1 -- 172.21.15.43:0/2234744034 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208007910 con 0x7fc2100b4140 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.388+0000 7fc215c00640 1 -- 172.21.15.43:0/2234744034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2100b4140 msgr2=0x7fc21016c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.388+0000 7fc215c00640 1 --2- 172.21.15.43:0/2234744034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2100b4140 0x7fc21016c280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc210074370 tx=0x7fc20802a610 comp rx=0 tx=0).stop 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 -- 172.21.15.43:0/2234744034 shutdown_connections 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 --2- 172.21.15.43:0/2234744034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2100b4140 0x7fc21016c280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 --2- 172.21.15.43:0/2234744034 >> v2:172.21.15.43:3300/0 conn(0x7fc2100b3460 0x7fc2100b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 -- 172.21.15.43:0/2234744034 >> 172.21.15.43:0/2234744034 conn(0x7fc2100b2900 msgr2=0x7fc2100b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:02.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 -- 172.21.15.43:0/2234744034 shutdown_connections 2024-04-26T20:26:02.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.389+0000 7fc215c00640 1 -- 172.21.15.43:0/2234744034 wait complete. 2024-04-26T20:26:02.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.391+0000 7fc215c00640 1 Processor -- start 2024-04-26T20:26:02.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.392+0000 7fc215c00640 1 -- start start 2024-04-26T20:26:02.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc2100b3460 0x7fc2100c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc21016bea0 0x7fc21016eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc21006f830 con 0x7fc2100b4140 2024-04-26T20:26:02.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc21006f310 con 0x7fc21016bea0 2024-04-26T20:26:02.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc215c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc21006f9b0 con 0x7fc2100b3460 2024-04-26T20:26:02.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.393+0000 7fc20e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55368/0 (socket says 172.21.15.43:55368) 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 -- 172.21.15.43:0/2744366135 learned_addr learned my addr 172.21.15.43:0/2744366135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 -- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc2100b3460 msgr2=0x7fc2100c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc2100b3460 0x7fc2100c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 -- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc21016bea0 msgr2=0x7fc21016eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc21016bea0 0x7fc21016eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.394+0000 7fc20e000640 1 -- 172.21.15.43:0/2744366135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2100c6270 con 0x7fc2100b4140 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc20e000640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800a7b0 tx=0x7fc1f800ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8004230 con 0x7fc2100b4140 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f800ae50 con 0x7fc2100b4140 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2100c6550 con 0x7fc2100b4140 2024-04-26T20:26:02.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800e960 con 0x7fc2100b4140 2024-04-26T20:26:02.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.395+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2100c6b10 con 0x7fc2100b4140 2024-04-26T20:26:02.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.396+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210069350 con 0x7fc2100b4140 2024-04-26T20:26:02.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.398+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1f8014050 con 0x7fc2100b4140 2024-04-26T20:26:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.399+0000 7fc206a00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 0x7fc1ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:02.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.400+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fc1f808c240 con 0x7fc2100b4140 2024-04-26T20:26:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.401+0000 7fc20ea00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 0x7fc1ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:02.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.402+0000 7fc20ea00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 0x7fc1ec06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2000018f0 tx=0x7fc200001e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:02.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.412+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1f805a340 con 0x7fc2100b4140 2024-04-26T20:26:02.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:02.667+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc210073b90 con 0x7fc2100b4140 2024-04-26T20:26:03.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.053+0000 7fc206a00640 1 -- 172.21.15.43:0/2744366135 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fc1f8059c40 con 0x7fc2100b4140 2024-04-26T20:26:03.054 INFO:teuthology.orchestra.run.smithi043.stderr:added data pool 4 to fsmap 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 msgr2=0x7fc1ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 0x7fc1ec06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2000018f0 tx=0x7fc200001e70 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 msgr2=0x7fc21008b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800a7b0 tx=0x7fc1f800ac70 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 shutdown_connections 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1ec06c8f0 0x7fc1ec06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc21016bea0 0x7fc21016eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2100b4140 0x7fc21008b920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 --2- 172.21.15.43:0/2744366135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc2100b3460 0x7fc2100c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 >> 172.21.15.43:0/2744366135 conn(0x7fc2100b2900 msgr2=0x7fc2100722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 shutdown_connections 2024-04-26T20:26:03.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.058+0000 7fc215c00640 1 -- 172.21.15.43:0/2744366135 wait complete. 2024-04-26T20:26:03.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:03.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.285+0000 7f7eca000640 1 Processor -- start 2024-04-26T20:26:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.285+0000 7f7eca000640 1 -- start start 2024-04-26T20:26:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.286+0000 7f7eca000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ec40af710 0x7f7ec40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.286+0000 7f7eca000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40cf600 0x7f7ec40cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.286+0000 7f7eca000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec4072db0 con 0x7f7ec415bb90 2024-04-26T20:26:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.286+0000 7f7eca000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec40cec10 con 0x7f7ec40af710 2024-04-26T20:26:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.286+0000 7f7eca000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec40b00b0 con 0x7f7ec40cf600 2024-04-26T20:26:03.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.287+0000 7f7ec3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ec40af710 0x7f7ec40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.287+0000 7f7ec3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ec40af710 0x7f7ec40afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:33420/0 (socket says 172.21.15.43:33420) 2024-04-26T20:26:03.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.287+0000 7f7ec3e00640 1 -- 172.21.15.43:0/3146401027 learned_addr learned my addr 172.21.15.43:0/3146401027 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:03.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.287+0000 7f7ec9600640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40cf600 0x7f7ec40cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.288+0000 7f7ec3e00640 1 -- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40cf600 msgr2=0x7f7ec40cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:03.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.288+0000 7f7ec3e00640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40cf600 0x7f7ec40cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.288+0000 7f7ec3e00640 1 -- 172.21.15.43:0/3146401027 >> v1:172.21.15.164:6789/0 conn(0x7f7ec415bb90 legacy=0x7f7ec40737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.288+0000 7f7ec3e00640 1 -- 172.21.15.43:0/3146401027 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec40cee40 con 0x7f7ec40af710 2024-04-26T20:26:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.290+0000 7f7ec3e00640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec40af710 0x7f7ec40afb70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009870 tx=0x7f7eb002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31ec2a46480e3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.290+0000 7f7ec3400640 1 -- 172.21.15.43:0/3146401027 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0037070 con 0x7f7ec40af710 2024-04-26T20:26:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.290+0000 7f7ec3400640 1 -- 172.21.15.43:0/3146401027 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb0032440 con 0x7f7ec40af710 2024-04-26T20:26:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.290+0000 7f7ec3400640 1 -- 172.21.15.43:0/3146401027 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0032810 con 0x7f7ec40af710 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 -- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec40af710 msgr2=0x7f7ec40afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec40af710 0x7f7ec40afb70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009870 tx=0x7f7eb002a270 comp rx=0 tx=0).stop 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 -- 172.21.15.43:0/3146401027 shutdown_connections 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40cf600 0x7f7ec40cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 --2- 172.21.15.43:0/3146401027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec40af710 0x7f7ec40afb70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 -- 172.21.15.43:0/3146401027 >> 172.21.15.43:0/3146401027 conn(0x7f7ec4068090 msgr2=0x7f7ec4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 -- 172.21.15.43:0/3146401027 shutdown_connections 2024-04-26T20:26:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.291+0000 7f7eca000640 1 -- 172.21.15.43:0/3146401027 wait complete. 2024-04-26T20:26:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.293+0000 7f7eca000640 1 Processor -- start 2024-04-26T20:26:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.293+0000 7f7eca000640 1 -- start start 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40af710 0x7f7ec41e7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 0x7f7ec415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4073eb0 con 0x7f7ec40cf600 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec40725a0 con 0x7f7ec415bb90 2024-04-26T20:26:03.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7eca000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec4072ad0 con 0x7f7ec40af710 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55378/0 (socket says 172.21.15.43:55378) 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7ec3e00640 1 -- 172.21.15.43:0/1441892014 learned_addr learned my addr 172.21.15.43:0/1441892014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7ec9600640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 0x7f7ec415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.294+0000 7f7ec8c00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40af710 0x7f7ec41e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 -- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40af710 msgr2=0x7f7ec41e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40af710 0x7f7ec41e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 -- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 msgr2=0x7f7ec415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 0x7f7ec415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 -- 172.21.15.43:0/1441892014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec41e7a60 con 0x7f7ec40cf600 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec9600640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 0x7f7ec415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:03.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec3e00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009990 tx=0x7f7eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0037040 con 0x7f7ec40cf600 2024-04-26T20:26:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec415c1a0 con 0x7f7ec40cf600 2024-04-26T20:26:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb0042070 con 0x7f7ec40cf600 2024-04-26T20:26:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.295+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0030940 con 0x7f7ec40cf600 2024-04-26T20:26:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.296+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec415c760 con 0x7f7ec40cf600 2024-04-26T20:26:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.297+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec4069350 con 0x7f7ec40cf600 2024-04-26T20:26:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.298+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eb0030ae0 con 0x7f7ec40cf600 2024-04-26T20:26:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.299+0000 7f7ec0c00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.299+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7eb00ac7c0 con 0x7f7ec40cf600 2024-04-26T20:26:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.299+0000 7f7ec8c00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.300+0000 7f7ec8c00640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 0x7f7ea406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007fd0 tx=0x7f7eb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.303+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7eb007a790 con 0x7f7ec40cf600 2024-04-26T20:26:03.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.552+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec40cce10 con 0x7f7ec40cf600 2024-04-26T20:26:03.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.554+0000 7f7ec0c00640 1 -- 172.21.15.43:0/1441892014 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f7eb007a090 con 0x7f7ec40cf600 2024-04-26T20:26:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:03.554 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:02.669472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:03.555 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 msgr2=0x7f7ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 0x7f7ea406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007fd0 tx=0x7f7eb8009040 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 msgr2=0x7f7ec415a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009990 tx=0x7f7eb00046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 shutdown_connections 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ec415bb90 0x7f7ec415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec40cf600 0x7f7ec415a1d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 --2- 172.21.15.43:0/1441892014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ec40af710 0x7f7ec41e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 >> 172.21.15.43:0/1441892014 conn(0x7f7ec4068090 msgr2=0x7f7ec4157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:03.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 shutdown_connections 2024-04-26T20:26:03.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.556+0000 7f7eca000640 1 -- 172.21.15.43:0/1441892014 wait complete. 2024-04-26T20:26:03.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.786+0000 7f65a2800640 1 Processor -- start 2024-04-26T20:26:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.786+0000 7f65a2800640 1 -- start start 2024-04-26T20:26:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f65a2800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f659c0d4df0 0x7f659c0d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f65a2800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0b4580 0x7f659c0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f65a2800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f659c0d3c60 con 0x7f659c15f3c0 2024-04-26T20:26:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f65a2800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f659c0b4aa0 con 0x7f659c0d4df0 2024-04-26T20:26:03.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f65a2800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f659c0d3f10 con 0x7f659c0b4580 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.787+0000 7f659b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f659c0d4df0 0x7f659c0d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f659c0d4df0 0x7f659c0d3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55380/0 (socket says 172.21.15.43:55380) 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 -- 172.21.15.43:0/4195951870 learned_addr learned my addr 172.21.15.43:0/4195951870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659be00640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0b4580 0x7f659c0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 -- 172.21.15.43:0/4195951870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0b4580 msgr2=0x7f659c0b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0b4580 0x7f659c0b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 -- 172.21.15.43:0/4195951870 >> v1:172.21.15.164:6789/0 conn(0x7f659c15f3c0 legacy=0x7f659c0d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:03.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.788+0000 7f659b400640 1 -- 172.21.15.43:0/4195951870 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0b4cf0 con 0x7f659c0d4df0 2024-04-26T20:26:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.789+0000 7f659b400640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c0d4df0 0x7f659c0d3720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f658c009870 tx=0x7f658c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4772e89d744125ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.789+0000 7f659a000640 1 -- 172.21.15.43:0/4195951870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c037070 con 0x7f659c0d4df0 2024-04-26T20:26:03.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.789+0000 7f659a000640 1 -- 172.21.15.43:0/4195951870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f658c032440 con 0x7f659c0d4df0 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.789+0000 7f659a000640 1 -- 172.21.15.43:0/4195951870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c032810 con 0x7f659c0d4df0 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 -- 172.21.15.43:0/4195951870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c0d4df0 msgr2=0x7f659c0d3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c0d4df0 0x7f659c0d3720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f658c009870 tx=0x7f658c02a210 comp rx=0 tx=0).stop 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 -- 172.21.15.43:0/4195951870 shutdown_connections 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0b4580 0x7f659c0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 --2- 172.21.15.43:0/4195951870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c0d4df0 0x7f659c0d3720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 -- 172.21.15.43:0/4195951870 >> 172.21.15.43:0/4195951870 conn(0x7f659c08ebd0 msgr2=0x7f659c09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 -- 172.21.15.43:0/4195951870 shutdown_connections 2024-04-26T20:26:03.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.790+0000 7f65a2800640 1 -- 172.21.15.43:0/4195951870 wait complete. 2024-04-26T20:26:03.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.792+0000 7f65a2800640 1 Processor -- start 2024-04-26T20:26:03.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.792+0000 7f65a2800640 1 -- start start 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f659c0b4580 0x7f659c1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0d4df0 0x7f659c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0b5d10 con 0x7f659c15f3c0 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0b52b0 con 0x7f659c0b4580 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f65a2800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f659c0b5e90 con 0x7f659c0d4df0 2024-04-26T20:26:03.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f659be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f659be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55384/0 (socket says 172.21.15.43:55384) 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f659be00640 1 -- 172.21.15.43:0/1195111204 learned_addr learned my addr 172.21.15.43:0/1195111204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.793+0000 7f659b400640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f659c0b4580 0x7f659c1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659aa00640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0d4df0 0x7f659c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 -- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0d4df0 msgr2=0x7f659c0b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0d4df0 0x7f659c0b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 -- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f659c0b4580 msgr2=0x7f659c1f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f659c0b4580 0x7f659c1f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 -- 172.21.15.43:0/1195111204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0cd440 con 0x7f659c15f3c0 2024-04-26T20:26:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f659be00640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6594007e70 tx=0x7f659400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594014620 con 0x7f659c15f3c0 2024-04-26T20:26:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6594019070 con 0x7f659c15f3c0 2024-04-26T20:26:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.794+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594014a20 con 0x7f659c15f3c0 2024-04-26T20:26:03.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.795+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c0cd720 con 0x7f659c15f3c0 2024-04-26T20:26:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.795+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c0cdce0 con 0x7f659c15f3c0 2024-04-26T20:26:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.796+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c069350 con 0x7f659c15f3c0 2024-04-26T20:26:03.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.797+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6594014bc0 con 0x7f659c15f3c0 2024-04-26T20:26:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.798+0000 7f6593400640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 0x7f657806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.798+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f659408c780 con 0x7f659c15f3c0 2024-04-26T20:26:03.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.798+0000 7f659b400640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 0x7f657806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:03.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.799+0000 7f659b400640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 0x7f657806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f658c034000 tx=0x7f658c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:03.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:03.802+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f659405a7d0 con 0x7f659c15f3c0 2024-04-26T20:26:04.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.010+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f659c1f53f0 con 0x7f659c15f3c0 2024-04-26T20:26:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.012+0000 7f6593400640 1 -- 172.21.15.43:0/1195111204 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f659405a0d0 con 0x7f659c15f3c0 2024-04-26T20:26:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:04.013 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.014+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 msgr2=0x7f657806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.014+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 0x7f657806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f658c034000 tx=0x7f658c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 msgr2=0x7f659c14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6594007e70 tx=0x7f659400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 shutdown_connections 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f657806c8f0 0x7f657806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f659c0d4df0 0x7f659c0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f659c15f3c0 0x7f659c14ea90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 --2- 172.21.15.43:0/1195111204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f659c0b4580 0x7f659c1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 >> 172.21.15.43:0/1195111204 conn(0x7f659c08ebd0 msgr2=0x7f659c06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 shutdown_connections 2024-04-26T20:26:04.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.015+0000 7f65a2800640 1 -- 172.21.15.43:0/1195111204 wait complete. 2024-04-26T20:26:04.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.235+0000 7feffbe00640 1 Processor -- start 2024-04-26T20:26:04.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.235+0000 7feffbe00640 1 -- start start 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.237+0000 7feffbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feffc0ce480 0x7feffc167b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.237+0000 7feffbe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feffc072a80 con 0x7feffc072620 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.237+0000 7feffbe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feffc0cd560 con 0x7feffc0cd180 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.237+0000 7feffbe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7feffc072d00 con 0x7feffc0ce480 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.238+0000 7feff2000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feffc072620 0x7feffc0cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51130/0 (socket says 172.21.15.43:51130) 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.238+0000 7feff2000640 1 -- 172.21.15.43:0/3946037018 learned_addr learned my addr 172.21.15.43:0/3946037018 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:04.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.238+0000 7feffb400640 1 --2- 172.21.15.43:0/3946037018 >> v2:172.21.15.164:3300/0 conn(0x7feffc0ce480 0x7feffc167b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.238+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599630933 0 0) 0x7fefe4001110 con 0x7feffc072620 2024-04-26T20:26:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.238+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefe0003520 con 0x7feffc072620 2024-04-26T20:26:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.239+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3663425956 0 0) 0x7fefe0003520 con 0x7feffc072620 2024-04-26T20:26:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.239+0000 7feffb400640 1 -- 172.21.15.43:0/3946037018 >> v1:172.21.15.164:6789/0 conn(0x7feffc0cd180 legacy=0x7feffc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.239+0000 7feffb400640 1 -- 172.21.15.43:0/3946037018 >> v1:172.21.15.43:6789/0 conn(0x7feffc072620 legacy=0x7feffc0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.239+0000 7feffb400640 1 -- 172.21.15.43:0/3946037018 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc072fe0 con 0x7feffc0ce480 2024-04-26T20:26:04.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.240+0000 7feffb400640 1 --2- 172.21.15.43:0/3946037018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc167b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feff4009d90 tx=0x7feff402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82978ee3cdf009c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.240+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4001840 con 0x7feffc0ce480 2024-04-26T20:26:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.240+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff4007540 con 0x7feffc0ce480 2024-04-26T20:26:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.240+0000 7feffa000640 1 -- 172.21.15.43:0/3946037018 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4007910 con 0x7feffc0ce480 2024-04-26T20:26:04.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 -- 172.21.15.43:0/3946037018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 msgr2=0x7feffc167b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 --2- 172.21.15.43:0/3946037018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc167b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feff4009d90 tx=0x7feff402a680 comp rx=0 tx=0).stop 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 -- 172.21.15.43:0/3946037018 shutdown_connections 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 --2- 172.21.15.43:0/3946037018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc167b60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 -- 172.21.15.43:0/3946037018 >> 172.21.15.43:0/3946037018 conn(0x7feffc068090 msgr2=0x7feffc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 -- 172.21.15.43:0/3946037018 shutdown_connections 2024-04-26T20:26:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.241+0000 7feffbe00640 1 -- 172.21.15.43:0/3946037018 wait complete. 2024-04-26T20:26:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.243+0000 7feffbe00640 1 Processor -- start 2024-04-26T20:26:04.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.243+0000 7feffbe00640 1 -- start start 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 0x7feffc15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feffc073da0 con 0x7feffc0cd180 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feffc0735a0 con 0x7feffc0ce480 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffbe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feffc06ead0 con 0x7feffc072620 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feff2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feff2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55400/0 (socket says 172.21.15.43:55400) 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feff2000640 1 -- 172.21.15.43:0/2914128530 learned_addr learned my addr 172.21.15.43:0/2914128530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffb400640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feff2000640 1 -- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 msgr2=0x7feffc15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.244+0000 7feffaa00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 0x7feffc15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff2000640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 0x7feffc15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff2000640 1 -- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 msgr2=0x7feffc0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff2000640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff2000640 1 -- 172.21.15.43:0/2914128530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc1f9360 con 0x7feffc0cd180 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feffb400640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feffaa00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 0x7feffc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff2000640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001550 tx=0x7fefe400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4017070 con 0x7feffc0cd180 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe4005af0 con 0x7feffc0cd180 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4007070 con 0x7feffc0cd180 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc1f9640 con 0x7feffc0cd180 2024-04-26T20:26:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.245+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc15c9c0 con 0x7feffc0cd180 2024-04-26T20:26:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.246+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc069350 con 0x7feffc0cd180 2024-04-26T20:26:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.247+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefe401a070 con 0x7feffc0cd180 2024-04-26T20:26:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.248+0000 7feff3e00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.248+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fefe408b410 con 0x7feffc0cd180 2024-04-26T20:26:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.249+0000 7feffaa00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.249+0000 7feffaa00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 0x7fefd806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fefec000f60 tx=0x7fefec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.253+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feffc072a80 con 0x7feffc0cd180 2024-04-26T20:26:04.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.449+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feffc0b5690 con 0x7feffc0cd180 2024-04-26T20:26:04.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.451+0000 7feff3e00640 1 -- 172.21.15.43:0/2914128530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7fefe40594f0 con 0x7feffc0cd180 2024-04-26T20:26:04.452 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:04.452 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:04.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 msgr2=0x7fefd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 0x7fefd806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fefec000f60 tx=0x7fefec009040 comp rx=0 tx=0).stop 2024-04-26T20:26:04.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 msgr2=0x7feffc1f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001550 tx=0x7fefe400cbe0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 shutdown_connections 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feffc0ce480 0x7feffc0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feffc0cd180 0x7feffc1f8a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 --2- 172.21.15.43:0/2914128530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feffc072620 0x7feffc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.453+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 >> 172.21.15.43:0/2914128530 conn(0x7feffc068090 msgr2=0x7feffc15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.454+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 shutdown_connections 2024-04-26T20:26:04.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.454+0000 7feffbe00640 1 -- 172.21.15.43:0/2914128530 wait complete. 2024-04-26T20:26:04.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.688+0000 7f851fe00640 1 Processor -- start 2024-04-26T20:26:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.689+0000 7f851fe00640 1 -- start start 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.690+0000 7f851fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85200af710 0x7f85200afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.690+0000 7f851fe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8520072db0 con 0x7f85200cf600 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.690+0000 7f851fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85200cee40 con 0x7f852015bb90 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.690+0000 7f851fe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f85200cec10 con 0x7f85200af710 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851e000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85200af710 0x7f85200afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851f400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85200cf600 0x7f85200cfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51134/0 (socket says 172.21.15.43:51134) 2024-04-26T20:26:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851f400640 1 -- 172.21.15.43:0/2915566829 learned_addr learned my addr 172.21.15.43:0/2915566829 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851e000640 1 -- 172.21.15.43:0/2915566829 >> v1:172.21.15.164:6789/0 conn(0x7f852015bb90 legacy=0x7f85200737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851e000640 1 -- 172.21.15.43:0/2915566829 >> v1:172.21.15.43:6789/0 conn(0x7f85200cf600 legacy=0x7f85200cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851e000640 1 -- 172.21.15.43:0/2915566829 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85200727f0 con 0x7f85200af710 2024-04-26T20:26:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.691+0000 7f851d600640 1 -- 172.21.15.43:0/2915566829 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247729818 0 0) 0x7f8520072db0 con 0x7f85200cf600 2024-04-26T20:26:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.692+0000 7f851e000640 1 --2- 172.21.15.43:0/2915566829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200af710 0x7f85200afb70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097a0 tx=0x7f850c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7750feabefbd645a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.692+0000 7f851d600640 1 -- 172.21.15.43:0/2915566829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c037070 con 0x7f85200af710 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851d600640 1 -- 172.21.15.43:0/2915566829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c032440 con 0x7f85200af710 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851d600640 1 -- 172.21.15.43:0/2915566829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c032810 con 0x7f85200af710 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 -- 172.21.15.43:0/2915566829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200af710 msgr2=0x7f85200afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 --2- 172.21.15.43:0/2915566829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200af710 0x7f85200afb70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097a0 tx=0x7f850c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 -- 172.21.15.43:0/2915566829 shutdown_connections 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 --2- 172.21.15.43:0/2915566829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200af710 0x7f85200afb70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 -- 172.21.15.43:0/2915566829 >> 172.21.15.43:0/2915566829 conn(0x7f8520068090 msgr2=0x7f8520068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 -- 172.21.15.43:0/2915566829 shutdown_connections 2024-04-26T20:26:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.693+0000 7f851fe00640 1 -- 172.21.15.43:0/2915566829 wait complete. 2024-04-26T20:26:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.695+0000 7f851fe00640 1 Processor -- start 2024-04-26T20:26:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.695+0000 7f851fe00640 1 -- start start 2024-04-26T20:26:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85200af710 0x7f85200ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f852015bb90 0x7f85200b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f85200725a0 con 0x7f85200cf600 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8520073eb0 con 0x7f85200af710 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f85200cf0e0 con 0x7f852015bb90 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55430/0 (socket says 172.21.15.43:55430) 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.696+0000 7f851e000640 1 -- 172.21.15.43:0/3643175308 learned_addr learned my addr 172.21.15.43:0/3643175308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851ea00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85200af710 0x7f85200ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851f400640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f852015bb90 0x7f85200b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 -- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f852015bb90 msgr2=0x7f85200b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f852015bb90 0x7f85200b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 -- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85200af710 msgr2=0x7f85200ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85200af710 0x7f85200ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 -- 172.21.15.43:0/3643175308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85200aeed0 con 0x7f85200cf600 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.697+0000 7f851e000640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098c0 tx=0x7f850c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.698+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c041070 con 0x7f85200cf600 2024-04-26T20:26:04.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.698+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c03c930 con 0x7f85200cf600 2024-04-26T20:26:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.698+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c037040 con 0x7f85200cf600 2024-04-26T20:26:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.698+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85201f6540 con 0x7f85200cf600 2024-04-26T20:26:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.698+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85201f6b00 con 0x7f85200cf600 2024-04-26T20:26:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.699+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8520069350 con 0x7f85200cf600 2024-04-26T20:26:04.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.700+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f850c044070 con 0x7f85200cf600 2024-04-26T20:26:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.701+0000 7f8516a00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.701+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f850c0ac9e0 con 0x7f85200cf600 2024-04-26T20:26:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.702+0000 7f851ea00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:04.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.702+0000 7f851ea00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 0x7f850006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:04.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.706+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f850c07aac0 con 0x7f85200cf600 2024-04-26T20:26:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.952+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85200af370 con 0x7f85200cf600 2024-04-26T20:26:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.953+0000 7f8516a00640 1 -- 172.21.15.43:0/3643175308 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f850c07a3c0 con 0x7f85200cf600 2024-04-26T20:26:04.954 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:04.954 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:02.669472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 msgr2=0x7f850006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 0x7f850006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510008040 comp rx=0 tx=0).stop 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 msgr2=0x7f85201505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098c0 tx=0x7f850c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 shutdown_connections 2024-04-26T20:26:04.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f850006c8f0 0x7f850006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f852015bb90 0x7f85200b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85200cf600 0x7f85201505b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.956+0000 7f851fe00640 1 --2- 172.21.15.43:0/3643175308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85200af710 0x7f85200ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.957+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 >> 172.21.15.43:0/3643175308 conn(0x7f8520068090 msgr2=0x7f85200b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.957+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 shutdown_connections 2024-04-26T20:26:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:04.957+0000 7f851fe00640 1 -- 172.21.15.43:0/3643175308 wait complete. 2024-04-26T20:26:04.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:05.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.183+0000 7fccf2c00640 1 Processor -- start 2024-04-26T20:26:05.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.183+0000 7fccf2c00640 1 -- start start 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.184+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec0737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf2c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccec0af710 0x7fccec0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf2c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccec072db0 con 0x7fccec15f610 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf2c00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fccec0cec10 con 0x7fccec0af710 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf2c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fccec0b00b0 con 0x7fccec15bb90 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccec0af710 0x7fccec0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccec0af710 0x7fccec0afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55434/0 (socket says 172.21.15.43:55434) 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 -- 172.21.15.43:0/2151443826 learned_addr learned my addr 172.21.15.43:0/2151443826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf1800640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec0737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 -- 172.21.15.43:0/2151443826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 msgr2=0x7fccec0737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec0737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 -- 172.21.15.43:0/2151443826 >> v1:172.21.15.164:6789/0 conn(0x7fccec15f610 legacy=0x7fccec15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf0e00640 1 -- 172.21.15.43:0/2151443826 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec0cee40 con 0x7fccec0af710 2024-04-26T20:26:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.185+0000 7fccf1800640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec0737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.186+0000 7fccf0e00640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec0afb70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009870 tx=0x7fcce002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81c07bd30135162 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccebe00640 1 -- 172.21.15.43:0/2151443826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0037070 con 0x7fccec0af710 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccebe00640 1 -- 172.21.15.43:0/2151443826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce0032440 con 0x7fccec0af710 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccebe00640 1 -- 172.21.15.43:0/2151443826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0032810 con 0x7fccec0af710 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 -- 172.21.15.43:0/2151443826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 msgr2=0x7fccec0afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec0afb70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009870 tx=0x7fcce002a210 comp rx=0 tx=0).stop 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 -- 172.21.15.43:0/2151443826 shutdown_connections 2024-04-26T20:26:05.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec0afb70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2151443826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec0737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.187+0000 7fccf2c00640 1 -- 172.21.15.43:0/2151443826 >> 172.21.15.43:0/2151443826 conn(0x7fccec068090 msgr2=0x7fccec068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.188+0000 7fccf2c00640 1 -- 172.21.15.43:0/2151443826 shutdown_connections 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.188+0000 7fccf2c00640 1 -- 172.21.15.43:0/2151443826 wait complete. 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.189+0000 7fccf2c00640 1 Processor -- start 2024-04-26T20:26:05.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.189+0000 7fccf2c00640 1 -- start start 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 0x7fccec15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0d0610 con 0x7fccec0af710 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0725a0 con 0x7fccec15f610 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf2c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fccec072ad0 con 0x7fccec15bb90 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.190+0000 7fccf1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55436/0 (socket says 172.21.15.43:55436) 2024-04-26T20:26:05.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 -- 172.21.15.43:0/2233399649 learned_addr learned my addr 172.21.15.43:0/2233399649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:05.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf2200640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 0x7fccec15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf0e00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 -- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 msgr2=0x7fccec15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 -- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 msgr2=0x7fccec15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 0x7fccec15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 -- 172.21.15.43:0/2233399649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec15e4d0 con 0x7fccec0af710 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf2200640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 0x7fccec15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fccf1800640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c7f0 tx=0x7fccdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.191+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc015070 con 0x7fccec0af710 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.192+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc007a30 con 0x7fccec0af710 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.192+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1569d0 con 0x7fccec0af710 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.192+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc007e00 con 0x7fccec0af710 2024-04-26T20:26:05.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.192+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec156f90 con 0x7fccec0af710 2024-04-26T20:26:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.193+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec069350 con 0x7fccec0af710 2024-04-26T20:26:05.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.194+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccdc018070 con 0x7fccec0af710 2024-04-26T20:26:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.195+0000 7fcce9600640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:05.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.195+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fccdc08be80 con 0x7fccec0af710 2024-04-26T20:26:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.195+0000 7fccf0e00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:05.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.196+0000 7fccf0e00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 0x7fccd006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcce0034000 tx=0x7fcce002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:05.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.199+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccdc059f60 con 0x7fccec0af710 2024-04-26T20:26:05.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.401+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccec15f380 con 0x7fccec0af710 2024-04-26T20:26:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.404+0000 7fcce9600640 1 -- 172.21.15.43:0/2233399649 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7fccdc059860 con 0x7fccec0af710 2024-04-26T20:26:05.406 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:05.406 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 msgr2=0x7fccd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 0x7fccd006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcce0034000 tx=0x7fcce002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 msgr2=0x7fccec145090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00c7f0 tx=0x7fccdc00cbf0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 shutdown_connections 2024-04-26T20:26:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fccd006c8f0 0x7fccd006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fccec15bb90 0x7fccec15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccec15f610 0x7fccec15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.406+0000 7fccf2c00640 1 --2- 172.21.15.43:0/2233399649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccec0af710 0x7fccec145090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.407+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 >> 172.21.15.43:0/2233399649 conn(0x7fccec068090 msgr2=0x7fccec0cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.407+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 shutdown_connections 2024-04-26T20:26:05.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:05.407+0000 7fccf2c00640 1 -- 172.21.15.43:0/2233399649 wait complete. 2024-04-26T20:26:05.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:25:57.479955+0000', 'modified': '2024-04-26T20:26:02.669472+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.164:6835/2023737548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6833', 'nonce': 2023737548}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2023737548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-26T20:26:05.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:26:05.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:26:06.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.054+0000 7fdc0ec00640 1 Processor -- start 2024-04-26T20:26:06.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.054+0000 7fdc0ec00640 1 -- start start 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc080d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc080d1a00 0x7fdc080b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 0x7fdc080b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdc080d0ce0 con 0x7fdc080d1a00 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdc080d0480 con 0x7fdc080b50b0 2024-04-26T20:26:06.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ec00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc080d0720 con 0x7fdc080d1620 2024-04-26T20:26:06.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ce00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc080d1a00 0x7fdc080b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ce00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc080d1a00 0x7fdc080b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37508/0 (socket says 172.21.15.43:37508) 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0ce00640 1 -- 172.21.15.43:0/1356581791 learned_addr learned my addr 172.21.15.43:0/1356581791 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0e200640 1 --2- 172.21.15.43:0/1356581791 >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 0x7fdc080b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.056+0000 7fdc0d800640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc080d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0ce00640 1 -- 172.21.15.43:0/1356581791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 msgr2=0x7fdc080d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc080d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0ce00640 1 -- 172.21.15.43:0/1356581791 >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 msgr2=0x7fdc080b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/1356581791 >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 0x7fdc080b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0ce00640 1 -- 172.21.15.43:0/1356581791 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc080d0a00 con 0x7fdc080d1a00 2024-04-26T20:26:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.057+0000 7fdc0e200640 1 --2- 172.21.15.43:0/1356581791 >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 0x7fdc080b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.058+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080b4b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc080bacb0 tx=0x7fdbf402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff892b7c8eeb6163 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.058+0000 7fdc07e00640 1 -- 172.21.15.43:0/1356581791 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4037070 con 0x7fdc080d1a00 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.058+0000 7fdc07e00640 1 -- 172.21.15.43:0/1356581791 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4032440 con 0x7fdc080d1a00 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.058+0000 7fdc07e00640 1 -- 172.21.15.43:0/1356581791 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4032810 con 0x7fdc080d1a00 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 -- 172.21.15.43:0/1356581791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 msgr2=0x7fdc080b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080b4b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc080bacb0 tx=0x7fdbf402a140 comp rx=0 tx=0).stop 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 -- 172.21.15.43:0/1356581791 shutdown_connections 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/1356581791 >> v2:172.21.15.164:3300/0 conn(0x7fdc080b50b0 0x7fdc080b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080b4b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/1356581791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc080d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 -- 172.21.15.43:0/1356581791 >> 172.21.15.43:0/1356581791 conn(0x7fdc080d4c30 msgr2=0x7fdc080d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 -- 172.21.15.43:0/1356581791 shutdown_connections 2024-04-26T20:26:06.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.059+0000 7fdc0ec00640 1 -- 172.21.15.43:0/1356581791 wait complete. 2024-04-26T20:26:06.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.061+0000 7fdc0ec00640 1 Processor -- start 2024-04-26T20:26:06.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.061+0000 7fdc0ec00640 1 -- start start 2024-04-26T20:26:06.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc080b50b0 0x7fdc080cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc0814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc080b5a80 con 0x7fdc080d1a00 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc080d1e60 con 0x7fdc080b50b0 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0ec00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc080b5c00 con 0x7fdc080d1620 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37512/0 (socket says 172.21.15.43:37512) 2024-04-26T20:26:06.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 -- 172.21.15.43:0/3637828114 learned_addr learned my addr 172.21.15.43:0/3637828114 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc080b50b0 0x7fdc080cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 -- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 msgr2=0x7fdc0814e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc0814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 -- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc080b50b0 msgr2=0x7fdc080cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc080b50b0 0x7fdc080cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.062+0000 7fdc0e200640 1 -- 172.21.15.43:0/3637828114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc080cd5f0 con 0x7fdc080d1a00 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc0e200640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc00007e40 tx=0x7fdc0000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc05600640 1 -- 172.21.15.43:0/3637828114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0001a070 con 0x7fdc080d1a00 2024-04-26T20:26:06.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc05600640 1 -- 172.21.15.43:0/3637828114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00003b90 con 0x7fdc080d1a00 2024-04-26T20:26:06.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc080c9b10 con 0x7fdc080d1a00 2024-04-26T20:26:06.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc05600640 1 -- 172.21.15.43:0/3637828114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000de80 con 0x7fdc080d1a00 2024-04-26T20:26:06.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.063+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc080ca0d0 con 0x7fdc080d1a00 2024-04-26T20:26:06.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.065+0000 7fdc05600640 1 -- 172.21.15.43:0/3637828114 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc0001d070 con 0x7fdc080d1a00 2024-04-26T20:26:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.067+0000 7fdc05600640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 0x7fdbe806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.067+0000 7fdc05600640 1 -- 172.21.15.43:0/3637828114 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdc0008c1d0 con 0x7fdc080d1a00 2024-04-26T20:26:06.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.067+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 0x7fdbe806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.068+0000 7fdc12fe9740 1 Processor -- start 2024-04-26T20:26:06.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.068+0000 7fdc12fe9740 1 -- start start 2024-04-26T20:26:06.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.068+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 0x7fdbe806ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006fd0 tx=0x7fdbfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80ae7b0 0x55c4d80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4d8012600 con 0x55c4d80d9270 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4d7faf100 con 0x55c4d80eae80 2024-04-26T20:26:06.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55c4d800e5c0 con 0x55c4d80ae7b0 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37528/0 (socket says 172.21.15.43:37528) 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 -- 172.21.15.43:0/254536381 learned_addr learned my addr 172.21.15.43:0/254536381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 -- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80ae7b0 msgr2=0x55c4d80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0d800640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80ae7b0 0x55c4d80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80ae7b0 0x55c4d80b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 -- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 msgr2=0x55c4d80aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0e200640 1 -- 172.21.15.43:0/254536381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d7d242d0 con 0x55c4d80d9270 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.069+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdc0e200640 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009700 tx=0x7fdc00059c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645f67add5a36fda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdbfa000640 1 -- 172.21.15.43:0/254536381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00057ea0 con 0x55c4d80d9270 2024-04-26T20:26:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdbfa000640 1 -- 172.21.15.43:0/254536381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00056210 con 0x55c4d80d9270 2024-04-26T20:26:06.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdbfa000640 1 -- 172.21.15.43:0/254536381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0005a000 con 0x55c4d80d9270 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdc12fe9740 1 -- 172.21.15.43:0/254536381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 msgr2=0x55c4d80d96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.070+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009700 tx=0x7fdc00059c60 comp rx=0 tx=0).stop 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 -- 172.21.15.43:0/254536381 shutdown_connections 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d96f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/254536381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80ae7b0 0x55c4d80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 -- 172.21.15.43:0/254536381 >> 172.21.15.43:0/254536381 conn(0x55c4d8042d10 msgr2=0x55c4d80f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 -- 172.21.15.43:0/254536381 shutdown_connections 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 -- 172.21.15.43:0/254536381 wait complete. 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.071+0000 7fdc12fe9740 1 -- start start 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.072+0000 7fdc12fe9740 1 Processor -- start 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4d7f99f70 con 0x55c4d80d9270 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4d80d6dc0 con 0x55c4d80ae7b0 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55c4d8105560 con 0x55c4d80eae80 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc12fe9740 10 client.0 _ll_register_callbacks cb 0x55c4d8008480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37540/0 (socket says 172.21.15.43:37540) 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3526663796 learned_addr learned my addr 172.21.15.43:0/3526663796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0e200640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 msgr2=0x55c4d80f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 msgr2=0x55c4d80dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.073+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3526663796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d7dee6b0 con 0x55c4d80d9270 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7539cfa877fb00b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdbeea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4041020 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc00056210 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdbf403c460 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fdbf404b050 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdc12fe9740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.074+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c4d80f0ac0 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00056210 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fdbf404e020 con 0x55c4d80d9270 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 10 client.4395 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 20 client.4395 populate_metadata read hostname 'smithi043' 2024-04-26T20:26:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ee900 0x55c4d81043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 4 client.4395 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.075+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 --> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d8020210 con 0x55c4d80ee900 2024-04-26T20:26:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.076+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ee900 0x55c4d81043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.076+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ee900 0x55c4d81043d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77e7d3fa4ab7272 server_cookie=a25cb82de10f3962 in_seq=0 out_seq=0 2024-04-26T20:26:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.076+0000 7fdbeea00640 10 client.4395 ms_handle_connect on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdbeea00640 1 -- 172.21.15.43:0/3526663796 <== mds.0 v2:172.21.15.164:6833/2023737548 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4d8020210 con 0x55c4d80ee900 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdbeea00640 10 client.4395 handle_command_reply: tid=0 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 client.4395 shutdown 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 msgr2=0x55c4d80d9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf40046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 shutdown_connections 2024-04-26T20:26:06.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ee900 0x55c4d81043d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80d9670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3526663796 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 >> 172.21.15.43:0/3526663796 conn(0x55c4d8042d10 msgr2=0x55c4d80d64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.078+0000 7fdbeea00640 0 client.4395 ms_handle_reset on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.079+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 shutdown_connections 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.079+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3526663796 wait complete. 2024-04-26T20:26:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.079+0000 7fdc12fe9740 20 client.4395 trim_cache size 0 max 16384 2024-04-26T20:26:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 Processor -- start 2024-04-26T20:26:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 -- start start 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80d9270 0x55c4d80dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc082019a0 con 0x55c4d80ae7b0 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08200d30 con 0x55c4d80eae80 2024-04-26T20:26:06.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.099+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc08201350 con 0x55c4d80d9270 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37542/0 (socket says 172.21.15.43:37542) 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 -- 172.21.15.43:0/3352010834 learned_addr learned my addr 172.21.15.43:0/3352010834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0e200640 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80d9270 0x55c4d80dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 -- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80d9270 msgr2=0x55c4d80dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80d9270 0x55c4d80dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 -- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 msgr2=0x55c4d80dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.100+0000 7fdc0d800640 1 -- 172.21.15.43:0/3352010834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d80acb10 con 0x55c4d80ae7b0 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00ebf0 tx=0x7fdbfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc7edfc1694c7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdbf9600640 1 -- 172.21.15.43:0/3352010834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc02d070 con 0x55c4d80ae7b0 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdbf9600640 1 -- 172.21.15.43:0/3352010834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00ece0 con 0x55c4d80ae7b0 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdbf9600640 1 -- 172.21.15.43:0/3352010834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00f040 con 0x55c4d80ae7b0 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3352010834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 msgr2=0x55c4d80a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00ebf0 tx=0x7fdbfc014000 comp rx=0 tx=0).stop 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3352010834 shutdown_connections 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80d9270 0x55c4d80dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80eae80 0x55c4d80dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3352010834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80ae7b0 0x55c4d80a88c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.101+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3352010834 >> 172.21.15.43:0/3352010834 conn(0x55c4d8042d10 msgr2=0x55c4d80af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.102+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3352010834 shutdown_connections 2024-04-26T20:26:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.102+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3352010834 wait complete. 2024-04-26T20:26:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.102+0000 7fdc12fe9740 1 -- start start 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.102+0000 7fdc12fe9740 1 Processor -- start 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80e9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c4d80ee780 con 0x55c4d80d9270 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c4d81053c0 con 0x55c4d80ae7b0 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55c4d80eed10 con 0x55c4d80eae80 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.103+0000 7fdc12fe9740 10 client.0 _ll_register_callbacks cb 0x7fdbf4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37552/0 (socket says 172.21.15.43:37552) 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3633929119 learned_addr learned my addr 172.21.15.43:0/3633929119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 msgr2=0x55c4d80f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 msgr2=0x55c4d80e9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80e9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 -- 172.21.15.43:0/3633929119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d80ea8a0 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc0ce00640 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf402a180 tx=0x7fdbf4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baec81730ea9dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdbeea00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4054060 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdbf4046690 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.104+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbfc00ece0 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fdbf405a050 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c4d8153e30 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc00ece0 con 0x55c4d80d9270 2024-04-26T20:26:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fdbf405d020 con 0x55c4d80d9270 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 20 client.4401 populate_metadata read hostname 'smithi043' 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ed270 0x55c4d80ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 4 client.4401 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:26:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.105+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 --> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] -- command(tid 0: {"prefix": "status"}) -- 0x55c4d801a080 con 0x55c4d80ed270 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.106+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ed270 0x55c4d80ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.106+0000 7fdc0d800640 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ed270 0x55c4d80ed650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f30a2a2e748cdc3 server_cookie=226a69e9e6f5c098 in_seq=0 out_seq=0 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.106+0000 7fdbeea00640 10 client.4401 ms_handle_connect on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdbeea00640 1 -- 172.21.15.43:0/3633929119 <== mds.0 v2:172.21.15.164:6833/2023737548 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c4d801a080 con 0x55c4d80ed270 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdbeea00640 10 client.4401 handle_command_reply: tid=0 2024-04-26T20:26:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 client.4401 shutdown 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 msgr2=0x55c4d80f0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf402a180 tx=0x7fdbf4008030 comp rx=0 tx=0).stop 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 shutdown_connections 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55c4d80ed270 0x55c4d80ed650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55c4d80eae80 0x55c4d80f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c4d80d9270 0x55c4d80f0ef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 --2- 172.21.15.43:0/3633929119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c4d80ae7b0 0x55c4d80e9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 >> 172.21.15.43:0/3633929119 conn(0x55c4d8042d10 msgr2=0x55c4d80af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdbeea00640 0 client.4401 ms_handle_reset on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 shutdown_connections 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.107+0000 7fdc12fe9740 1 -- 172.21.15.43:0/3633929119 wait complete. 2024-04-26T20:26:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc12fe9740 20 client.4401 trim_cache size 0 max 16384 2024-04-26T20:26:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:26:06.113 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6a354559-7839-4522-9cd7-4f1548bb621f", 2024-04-26T20:26:06.113 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-26T20:26:06.113 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4273, 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 8.1426146599999996, 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 7, 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 24, 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 16.264403966 2024-04-26T20:26:06.114 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 msgr2=0x7fdbe806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 0x7fdbe806ee00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc006fd0 tx=0x7fdbfc008040 comp rx=0 tx=0).stop 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 msgr2=0x7fdc080cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc00007e40 tx=0x7fdc0000d4a0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 shutdown_connections 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbe806c940 0x7fdbe806ee00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc080d1620 0x7fdc0814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc080d1a00 0x7fdc080cabb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 --2- 172.21.15.43:0/3637828114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc080b50b0 0x7fdc080cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 >> 172.21.15.43:0/3637828114 conn(0x7fdc080d4c30 msgr2=0x7fdc080d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 shutdown_connections 2024-04-26T20:26:06.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.108+0000 7fdc0ec00640 1 -- 172.21.15.43:0/3637828114 wait complete. 2024-04-26T20:26:06.128 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:26:06.129 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:06.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:26:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.337+0000 7f5105600640 1 Processor -- start 2024-04-26T20:26:06.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.338+0000 7f5105600640 1 -- start start 2024-04-26T20:26:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.339+0000 7f5105600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5100146630 0x7f5100161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.339+0000 7f5105600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51000d0dc0 0x7f51000d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.339+0000 7f5105600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5100161a70 con 0x7f5100071ea0 2024-04-26T20:26:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.339+0000 7f5105600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f51000d1760 con 0x7f5100146630 2024-04-26T20:26:06.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.339+0000 7f5105600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51000d0640 con 0x7f51000d0dc0 2024-04-26T20:26:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5100146630 0x7f5100161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50ff400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5100071ea0 0x7f5100072310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54280/0 (socket says 172.21.15.43:54280) 2024-04-26T20:26:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50ff400640 1 -- 172.21.15.43:0/283609034 learned_addr learned my addr 172.21.15.43:0/283609034 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fe000640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51000d0dc0 0x7f51000d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fd600640 1 -- 172.21.15.43:0/283609034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359258316 0 0) 0x7f50f8001490 con 0x7f5100071ea0 2024-04-26T20:26:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fea00640 1 -- 172.21.15.43:0/283609034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51000d0dc0 msgr2=0x7f51000d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fea00640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51000d0dc0 0x7f51000d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fea00640 1 -- 172.21.15.43:0/283609034 >> v1:172.21.15.43:6789/0 conn(0x7f5100071ea0 legacy=0x7f5100072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.340+0000 7f50fea00640 1 -- 172.21.15.43:0/283609034 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510015d880 con 0x7f5100146630 2024-04-26T20:26:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.342+0000 7f50fea00640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100146630 0x7f5100161530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097a0 tx=0x7f50f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34287ac64a87c9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.342+0000 7f50fd600640 1 -- 172.21.15.43:0/283609034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0037070 con 0x7f5100146630 2024-04-26T20:26:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.342+0000 7f50fd600640 1 -- 172.21.15.43:0/283609034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0032440 con 0x7f5100146630 2024-04-26T20:26:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.342+0000 7f50fd600640 1 -- 172.21.15.43:0/283609034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0032810 con 0x7f5100146630 2024-04-26T20:26:06.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 -- 172.21.15.43:0/283609034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100146630 msgr2=0x7f5100161530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100146630 0x7f5100161530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097a0 tx=0x7f50f002a180 comp rx=0 tx=0).stop 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 -- 172.21.15.43:0/283609034 shutdown_connections 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51000d0dc0 0x7f51000d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 --2- 172.21.15.43:0/283609034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100146630 0x7f5100161530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 -- 172.21.15.43:0/283609034 >> 172.21.15.43:0/283609034 conn(0x7f51000682a0 msgr2=0x7f51000686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 -- 172.21.15.43:0/283609034 shutdown_connections 2024-04-26T20:26:06.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.343+0000 7f5105600640 1 -- 172.21.15.43:0/283609034 wait complete. 2024-04-26T20:26:06.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.345+0000 7f5105600640 1 Processor -- start 2024-04-26T20:26:06.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.345+0000 7f5105600640 1 -- start start 2024-04-26T20:26:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51000d0dc0 0x7f51000c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5100146630 0x7f510006e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f510015e120 con 0x7f5100071ea0 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f510015de40 con 0x7f51000d0dc0 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f5105600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5100146a10 con 0x7f5100146630 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37572/0 (socket says 172.21.15.43:37572) 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50fea00640 1 -- 172.21.15.43:0/4049032715 learned_addr learned my addr 172.21.15.43:0/4049032715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50fe000640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51000d0dc0 0x7f51000c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50ff400640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5100146630 0x7f510006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.346+0000 7f50fea00640 1 -- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5100146630 msgr2=0x7f510006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50fea00640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5100146630 0x7f510006e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50fea00640 1 -- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51000d0dc0 msgr2=0x7f51000c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50fea00640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51000d0dc0 0x7f51000c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50fea00640 1 -- 172.21.15.43:0/4049032715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51001f5d00 con 0x7f5100071ea0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50fea00640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50f00098c0 tx=0x7f50f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0037040 con 0x7f5100071ea0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0030770 con 0x7f5100071ea0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030b40 con 0x7f5100071ea0 2024-04-26T20:26:06.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51001f5fe0 con 0x7f5100071ea0 2024-04-26T20:26:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.347+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51001f65a0 con 0x7f5100071ea0 2024-04-26T20:26:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.348+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5100069350 con 0x7f5100071ea0 2024-04-26T20:26:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.349+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50f0044020 con 0x7f5100071ea0 2024-04-26T20:26:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.350+0000 7f50f6a00640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 0x7f50dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:06.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.350+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f50f00ac3c0 con 0x7f5100071ea0 2024-04-26T20:26:06.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.351+0000 7f50fe000640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 0x7f50dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:06.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.352+0000 7f50fe000640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 0x7f50dc06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f510006fca0 tx=0x7f50e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:06.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.355+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f50f0032e20 con 0x7f5100071ea0 2024-04-26T20:26:06.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.606+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5100073cc0 con 0x7f5100071ea0 2024-04-26T20:26:06.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.607+0000 7f50f6a00640 1 -- 172.21.15.43:0/4049032715 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f50f007a4a0 con 0x7f5100071ea0 2024-04-26T20:26:06.608 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:26:06.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.609+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 msgr2=0x7f50dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.609+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 0x7f50dc06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f510006fca0 tx=0x7f50e8008040 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.609+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 msgr2=0x7f51000c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.609+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50f00098c0 tx=0x7f50f00046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 shutdown_connections 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f50dc06c8f0 0x7f50dc06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5100146630 0x7f510006e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51000d0dc0 0x7f51000c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 --2- 172.21.15.43:0/4049032715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5100071ea0 0x7f51000c5a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 >> 172.21.15.43:0/4049032715 conn(0x7f51000682a0 msgr2=0x7f51000b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 shutdown_connections 2024-04-26T20:26:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06.610+0000 7f5105600640 1 -- 172.21.15.43:0/4049032715 wait complete. 2024-04-26T20:26:06.630 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:26:06.630 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:26:06.630 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:26:06.631 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:26:06.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:06.631 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-26T20:26:06.682 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:06.682 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:06.682 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-26T20:26:06.683 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42625sec preferred_lft 42625sec 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:06.684 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-26T20:26:06.685 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:26:06.685 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:06.685 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:06.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:26:06.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-26T20:26:06.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-26T20:26:06.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:26:06.686 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:06.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:07.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:07.185 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:26:07.257 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-26T20:26:07.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:07.261 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-26T20:26:07.309 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-26T20:26:07.309 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:26:07.309 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:26:07.309 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:26:07.309 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:26:07.310 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:07.310 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:07.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:26:07.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:26:07.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:26:07.311 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:07.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:08.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:08.024 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:26:08.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:08.069 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-26T20:26:08.149 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:08.149 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:08.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:08.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T20:26:08.149 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:08.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:08.375 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T20:26:08.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T20:26:08.376 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:08.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:08.661 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:08.661 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:08.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-26T20:26:08.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:26:08.661 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:08.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:08.865 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T20:26:08.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:08.865 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:08.908 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:26:08.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:08.908 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:08.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:09.069 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:09.112 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:09.112 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:09.112 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:09.112 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:09.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:09.115 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:09.115 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:09.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:09.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:09.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:09.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:09.116 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:09.212 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:26:09.212 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T20:26:09.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:09.289 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:09.329 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:09.330 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:09.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:09.332 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:09.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:2024-04-26T20:26:09.399+0000 7f5943c29480 -1 init, newargv = 0x55d9aea91b90 newargc=15 2024-04-26T20:26:09.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[38830]: starting ceph client 2024-04-26T20:26:09.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[38830]: starting fuse 2024-04-26T20:26:09.417 INFO:teuthology.orchestra.run.smithi043.stdout:42 2024-04-26T20:26:09.417 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-26T20:26:09.417 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-26T20:26:09.418 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-26T20:26:09.419 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:26:09.419 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:09.419 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.admin")) 2024-04-26T20:26:09.419 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-26T20:26:09.617 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.admin.38830.asok 2024-04-26T20:26:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:09.635 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.38830.asok 2024-04-26T20:26:09.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:09.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.38830.asok status 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "admin", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "38830", 2024-04-26T20:26:09.880 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4416, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4416 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3515328873 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:09.881 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3515328873 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4416 172.21.15.43:0/3515328873", 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "172.21.15.43:0/3515328873", 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 7, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 24, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-26T20:26:09.882 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:26:09.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:09.895 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-26T20:26:09.895 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:09.952 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-26T20:26:09.952 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:09.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:09.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:10.041 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T20:26:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:26:10.203 INFO:teuthology.orchestra.run.smithi043.stdout:# file: . 2024-04-26T20:26:10.203 INFO:teuthology.orchestra.run.smithi043.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T20:26:10.203 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:10.214 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:26:10.214 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:10.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:10.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.484+0000 7f19f0a00640 1 Processor -- start 2024-04-26T20:26:10.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.484+0000 7f19f0a00640 1 -- start start 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f19ec0cd180 0x7f19ec072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19ec15dc50 0x7f19ec0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec0ce2b0 0x7f19ec1607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0ce720 con 0x7f19ec15dc50 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0cd560 con 0x7f19ec0cd180 2024-04-26T20:26:10.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19f0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec072d00 con 0x7f19ec0ce2b0 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19e9e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19ec15dc50 0x7f19ec0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19e9e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19ec15dc50 0x7f19ec0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37578/0 (socket says 172.21.15.43:37578) 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.486+0000 7f19e9e00640 1 -- 172.21.15.43:0/1049220897 learned_addr learned my addr 172.21.15.43:0/1049220897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19eb200640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec0ce2b0 0x7f19ec1607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19ea800640 1 --2- 172.21.15.43:0/1049220897 >> v2:172.21.15.164:3300/0 conn(0x7f19ec0cd180 0x7f19ec072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19e9e00640 1 -- 172.21.15.43:0/1049220897 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec0ce2b0 msgr2=0x7f19ec1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19e9e00640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec0ce2b0 0x7f19ec1607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19e9e00640 1 -- 172.21.15.43:0/1049220897 >> v2:172.21.15.164:3300/0 conn(0x7f19ec0cd180 msgr2=0x7f19ec072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19e9e00640 1 --2- 172.21.15.43:0/1049220897 >> v2:172.21.15.164:3300/0 conn(0x7f19ec0cd180 0x7f19ec072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.487+0000 7f19e9e00640 1 -- 172.21.15.43:0/1049220897 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec072fe0 con 0x7f19ec15dc50 2024-04-26T20:26:10.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.488+0000 7f19e9e00640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec15dc50 0x7f19ec0cdd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0717f0 tx=0x7f19d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4d6daf2384fe7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.488+0000 7f19e9400640 1 -- 172.21.15.43:0/1049220897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8037070 con 0x7f19ec15dc50 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19e9400640 1 -- 172.21.15.43:0/1049220897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d8032440 con 0x7f19ec15dc50 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19e9400640 1 -- 172.21.15.43:0/1049220897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8032810 con 0x7f19ec15dc50 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 -- 172.21.15.43:0/1049220897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec15dc50 msgr2=0x7f19ec0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec15dc50 0x7f19ec0cdd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0717f0 tx=0x7f19d802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 -- 172.21.15.43:0/1049220897 shutdown_connections 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec0ce2b0 0x7f19ec1607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 --2- 172.21.15.43:0/1049220897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec15dc50 0x7f19ec0cdd70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 --2- 172.21.15.43:0/1049220897 >> v2:172.21.15.164:3300/0 conn(0x7f19ec0cd180 0x7f19ec072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 -- 172.21.15.43:0/1049220897 >> 172.21.15.43:0/1049220897 conn(0x7f19ec068090 msgr2=0x7f19ec068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 -- 172.21.15.43:0/1049220897 shutdown_connections 2024-04-26T20:26:10.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.489+0000 7f19f0a00640 1 -- 172.21.15.43:0/1049220897 wait complete. 2024-04-26T20:26:10.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.491+0000 7f19f0a00640 1 Processor -- start 2024-04-26T20:26:10.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.491+0000 7f19f0a00640 1 -- start start 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19ec0ce2b0 0x7f19ec140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 0x7f19ec15f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec073d80 con 0x7f19ec0cd180 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec0735a0 con 0x7f19ec0ce2b0 2024-04-26T20:26:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.492+0000 7f19f0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec15e0d0 con 0x7f19ec15dc50 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37592/0 (socket says 172.21.15.43:37592) 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 -- 172.21.15.43:0/2476950943 learned_addr learned my addr 172.21.15.43:0/2476950943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19e9e00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19ec0ce2b0 0x7f19ec140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19eb200640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 0x7f19ec15f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 -- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 msgr2=0x7f19ec15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 0x7f19ec15f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 -- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19ec0ce2b0 msgr2=0x7f19ec140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19ec0ce2b0 0x7f19ec140020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 -- 172.21.15.43:0/2476950943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec1f4dc0 con 0x7f19ec0cd180 2024-04-26T20:26:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19eb200640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 0x7f19ec15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.493+0000 7f19ea800640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d830 tx=0x7f19dc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.494+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc018030 con 0x7f19ec0cd180 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.494+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc00be10 con 0x7f19ec0cd180 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.494+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0098b0 con 0x7f19ec0cd180 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.494+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec140a90 con 0x7f19ec0cd180 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.494+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec13d190 con 0x7f19ec0cd180 2024-04-26T20:26:10.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.495+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec069350 con 0x7f19ec0cd180 2024-04-26T20:26:10.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.496+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19dc01b070 con 0x7f19ec0cd180 2024-04-26T20:26:10.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.497+0000 7f19e2a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 0x7f19cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.497+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f19dc08c080 con 0x7f19ec0cd180 2024-04-26T20:26:10.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.497+0000 7f19e9e00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 0x7f19cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.498+0000 7f19e9e00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 0x7f19cc06ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19d8034000 tx=0x7f19d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.502+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19dc05a0d0 con 0x7f19ec0cd180 2024-04-26T20:26:10.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.709+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19ec0d0670 con 0x7f19ec0cd180 2024-04-26T20:26:10.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.712+0000 7f19e2a00640 1 -- 172.21.15.43:0/2476950943 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f19dc0599d0 con 0x7f19ec0cd180 2024-04-26T20:26:10.714 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:10.714 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:10.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.714+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 msgr2=0x7f19cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.714+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 0x7f19cc06ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19d8034000 tx=0x7f19d802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.714+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 msgr2=0x7f19ec13c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.714+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d830 tx=0x7f19dc00dcf0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 shutdown_connections 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f19cc06c920 0x7f19cc06ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19ec15dc50 0x7f19ec15f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19ec0ce2b0 0x7f19ec140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 --2- 172.21.15.43:0/2476950943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19ec0cd180 0x7f19ec13c5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 >> 172.21.15.43:0/2476950943 conn(0x7f19ec068090 msgr2=0x7f19ec0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 shutdown_connections 2024-04-26T20:26:10.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.715+0000 7f19f0a00640 1 -- 172.21.15.43:0/2476950943 wait complete. 2024-04-26T20:26:10.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:10.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:10.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.954+0000 7fcaae800640 1 Processor -- start 2024-04-26T20:26:10.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.954+0000 7fcaae800640 1 -- start start 2024-04-26T20:26:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.955+0000 7fcaae800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa81604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.955+0000 7fcaae800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcaa806df40 0x7fcaa80b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaae800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa80b3d80 con 0x7fcaa80b52e0 2024-04-26T20:26:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaae800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa80b5730 con 0x7fcaa806df40 2024-04-26T20:26:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaae800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa80b4060 con 0x7fcaa81458e0 2024-04-26T20:26:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaa7e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcaa80b52e0 0x7fcaa815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54294/0 (socket says 172.21.15.43:54294) 2024-04-26T20:26:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaa7e00640 1 -- 172.21.15.43:0/878220254 learned_addr learned my addr 172.21.15.43:0/878220254 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:10.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaa6a00640 1 --2- 172.21.15.43:0/878220254 >> v2:172.21.15.164:3300/0 conn(0x7fcaa806df40 0x7fcaa80b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.956+0000 7fcaa7400640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa81604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553181746 0 0) 0x7fcaa80b3d80 con 0x7fcaa80b52e0 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca8c003520 con 0x7fcaa80b52e0 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6a00640 1 -- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 msgr2=0x7fcaa81604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6a00640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa81604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116637248 0 0) 0x7fca8c003520 con 0x7fcaa80b52e0 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6a00640 1 -- 172.21.15.43:0/878220254 >> v1:172.21.15.43:6789/0 conn(0x7fcaa80b52e0 legacy=0x7fcaa815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.957+0000 7fcaa6a00640 1 -- 172.21.15.43:0/878220254 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80b4340 con 0x7fcaa806df40 2024-04-26T20:26:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.958+0000 7fcaa6a00640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa806df40 0x7fcaa80b4da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca940097a0 tx=0x7fca9402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3acb2b2ef60584c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.959+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94037070 con 0x7fcaa806df40 2024-04-26T20:26:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.959+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca9402ab10 con 0x7fcaa806df40 2024-04-26T20:26:10.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.959+0000 7fcaa6000640 1 -- 172.21.15.43:0/878220254 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00019f0 con 0x7fcaa806df40 2024-04-26T20:26:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.959+0000 7fcaae800640 1 -- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa806df40 msgr2=0x7fcaa80b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.959+0000 7fcaae800640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa806df40 0x7fcaa80b4da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca940097a0 tx=0x7fca9402a080 comp rx=0 tx=0).stop 2024-04-26T20:26:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 -- 172.21.15.43:0/878220254 shutdown_connections 2024-04-26T20:26:10.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa806df40 0x7fcaa80b4da0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 --2- 172.21.15.43:0/878220254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa81604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 -- 172.21.15.43:0/878220254 >> 172.21.15.43:0/878220254 conn(0x7fcaa80b2900 msgr2=0x7fcaa80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 -- 172.21.15.43:0/878220254 shutdown_connections 2024-04-26T20:26:10.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.960+0000 7fcaae800640 1 -- 172.21.15.43:0/878220254 wait complete. 2024-04-26T20:26:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.962+0000 7fcaae800640 1 Processor -- start 2024-04-26T20:26:10.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.962+0000 7fcaae800640 1 -- start start 2024-04-26T20:26:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 0x7fcaa8136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa815e710 con 0x7fcaa80af4d0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa80b4900 con 0x7fcaa80b52e0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaae800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa815e890 con 0x7fcaa81458e0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37600/0 (socket says 172.21.15.43:37600) 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaa7e00640 1 -- 172.21.15.43:0/3683058916 learned_addr learned my addr 172.21.15.43:0/3683058916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaa6a00640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.963+0000 7fcaa7400640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 0x7fcaa8136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 -- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 msgr2=0x7fcaa80aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa80aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 -- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 msgr2=0x7fcaa8136580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 0x7fcaa8136580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 -- 172.21.15.43:0/3683058916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa8136ef0 con 0x7fcaa80af4d0 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7400640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 0x7fcaa8136580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:10.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaa7e00640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000cc00 tx=0x7fcaa000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0017070 con 0x7fcaa80af4d0 2024-04-26T20:26:10.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa000ce10 con 0x7fcaa80af4d0 2024-04-26T20:26:10.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0012600 con 0x7fcaa80af4d0 2024-04-26T20:26:10.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.964+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa81371d0 con 0x7fcaa80af4d0 2024-04-26T20:26:10.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.965+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa8134900 con 0x7fcaa80af4d0 2024-04-26T20:26:10.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.966+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa8069350 con 0x7fcaa80af4d0 2024-04-26T20:26:10.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.967+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaa001a070 con 0x7fcaa80af4d0 2024-04-26T20:26:10.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.968+0000 7fca9f400640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 0x7fca8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:10.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.969+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcaa008bc90 con 0x7fcaa80af4d0 2024-04-26T20:26:10.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.969+0000 7fcaa7400640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 0x7fca8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:10.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.970+0000 7fcaa7400640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 0x7fca8806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca98009b90 tx=0x7fca98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:10.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:10.973+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcaa0059720 con 0x7fcaa80af4d0 2024-04-26T20:26:11.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.182+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcaa8074170 con 0x7fcaa80af4d0 2024-04-26T20:26:11.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.183+0000 7fca9f400640 1 -- 172.21.15.43:0/3683058916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fcaa005da20 con 0x7fcaa80af4d0 2024-04-26T20:26:11.185 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 msgr2=0x7fca8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 0x7fca8806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca98009b90 tx=0x7fca98008040 comp rx=0 tx=0).stop 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 msgr2=0x7fcaa80729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000cc00 tx=0x7fcaa000a430 comp rx=0 tx=0).stop 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 shutdown_connections 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca8806c8f0 0x7fca8806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcaa81458e0 0x7fcaa80aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcaa80af4d0 0x7fcaa80729e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 --2- 172.21.15.43:0/3683058916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcaa80b52e0 0x7fcaa8136580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.186+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 >> 172.21.15.43:0/3683058916 conn(0x7fcaa80b2900 msgr2=0x7fcaa80d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.187+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 shutdown_connections 2024-04-26T20:26:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.187+0000 7fcaae800640 1 -- 172.21.15.43:0/3683058916 wait complete. 2024-04-26T20:26:11.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.421+0000 7fba17400640 1 Processor -- start 2024-04-26T20:26:11.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.421+0000 7fba17400640 1 -- start start 2024-04-26T20:26:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba17400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba100d1f40 0x7fba1016a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba17400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba100d12e0 con 0x7fba100cfc30 2024-04-26T20:26:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba17400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba100d0240 con 0x7fba1015cca0 2024-04-26T20:26:11.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba17400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fba100d0520 con 0x7fba100d1f40 2024-04-26T20:26:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba15400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba100d1f40 0x7fba1016a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba15400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba100d1f40 0x7fba1016a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37612/0 (socket says 172.21.15.43:37612) 2024-04-26T20:26:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba15400640 1 -- 172.21.15.43:0/3207451148 learned_addr learned my addr 172.21.15.43:0/3207451148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.423+0000 7fba0fe00640 1 --1- 172.21.15.43:0/3207451148 >> v1:172.21.15.43:6789/0 conn(0x7fba100cfc30 0x7fba100d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54308/0 (socket says 172.21.15.43:54308) 2024-04-26T20:26:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.424+0000 7fba15400640 1 -- 172.21.15.43:0/3207451148 >> v1:172.21.15.164:6789/0 conn(0x7fba1015cca0 legacy=0x7fba10160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.424+0000 7fba15400640 1 -- 172.21.15.43:0/3207451148 >> v1:172.21.15.43:6789/0 conn(0x7fba100cfc30 legacy=0x7fba100d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.424+0000 7fba15400640 1 -- 172.21.15.43:0/3207451148 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba100d0800 con 0x7fba100d1f40 2024-04-26T20:26:11.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.425+0000 7fba15400640 1 --2- 172.21.15.43:0/3207451148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100d1f40 0x7fba1016a140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba08009e50 tx=0x7fba0802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb28efc00c886143 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.425+0000 7fba0f400640 1 -- 172.21.15.43:0/3207451148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08036070 con 0x7fba100d1f40 2024-04-26T20:26:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.425+0000 7fba0f400640 1 -- 172.21.15.43:0/3207451148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba08031600 con 0x7fba100d1f40 2024-04-26T20:26:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.425+0000 7fba0f400640 1 -- 172.21.15.43:0/3207451148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba080319d0 con 0x7fba100d1f40 2024-04-26T20:26:11.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 -- 172.21.15.43:0/3207451148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100d1f40 msgr2=0x7fba1016a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 --2- 172.21.15.43:0/3207451148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100d1f40 0x7fba1016a140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba08009e50 tx=0x7fba0802a760 comp rx=0 tx=0).stop 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 -- 172.21.15.43:0/3207451148 shutdown_connections 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 --2- 172.21.15.43:0/3207451148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100d1f40 0x7fba1016a140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 -- 172.21.15.43:0/3207451148 >> 172.21.15.43:0/3207451148 conn(0x7fba100d4e40 msgr2=0x7fba100d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 -- 172.21.15.43:0/3207451148 shutdown_connections 2024-04-26T20:26:11.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.426+0000 7fba17400640 1 -- 172.21.15.43:0/3207451148 wait complete. 2024-04-26T20:26:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.428+0000 7fba17400640 1 Processor -- start 2024-04-26T20:26:11.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.428+0000 7fba17400640 1 -- start start 2024-04-26T20:26:11.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fba100d1f40 0x7fba1015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba1015cca0 0x7fba100b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba100d3a20 con 0x7fba100cfc30 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba100d0dc0 con 0x7fba1015cca0 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba17400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fba100d3ba0 con 0x7fba100d1f40 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba14a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba14a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37622/0 (socket says 172.21.15.43:37622) 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba14a00640 1 -- 172.21.15.43:0/42271188 learned_addr learned my addr 172.21.15.43:0/42271188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba15400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba1015cca0 0x7fba100b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.429+0000 7fba0fe00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fba100d1f40 0x7fba1015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 -- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fba100d1f40 msgr2=0x7fba1015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fba100d1f40 0x7fba1015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 -- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba1015cca0 msgr2=0x7fba100b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba1015cca0 0x7fba100b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 -- 172.21.15.43:0/42271188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba100b89d0 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba14a00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c430 tx=0x7fba0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04011790 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04017070 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04011bb0 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.430+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba100b8d10 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.431+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba100b7050 con 0x7fba100cfc30 2024-04-26T20:26:11.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.432+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba10069350 con 0x7fba100cfc30 2024-04-26T20:26:11.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.433+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba0401c460 con 0x7fba100cfc30 2024-04-26T20:26:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.434+0000 7fba0cc00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 0x7fb9f006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.434+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fba0408bcd0 con 0x7fba100cfc30 2024-04-26T20:26:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.434+0000 7fba0fe00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 0x7fb9f006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.435+0000 7fba0fe00640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 0x7fb9f006ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba1006e440 tx=0x7fb9fc007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.439+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba100b8d10 con 0x7fba100cfc30 2024-04-26T20:26:11.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.651+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba100c84d0 con 0x7fba100cfc30 2024-04-26T20:26:11.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.652+0000 7fba0cc00640 1 -- 172.21.15.43:0/42271188 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fba04059e30 con 0x7fba100cfc30 2024-04-26T20:26:11.653 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:11.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 msgr2=0x7fb9f006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 0x7fb9f006ee00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba1006e440 tx=0x7fb9fc007ce0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 msgr2=0x7fba100b6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c430 tx=0x7fba0400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 shutdown_connections 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb9f006c940 0x7fb9f006ee00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fba100d1f40 0x7fba1015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba1015cca0 0x7fba100b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 --2- 172.21.15.43:0/42271188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba100cfc30 0x7fba100b6b10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 >> 172.21.15.43:0/42271188 conn(0x7fba100d4e40 msgr2=0x7fba100b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 shutdown_connections 2024-04-26T20:26:11.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.655+0000 7fba17400640 1 -- 172.21.15.43:0/42271188 wait complete. 2024-04-26T20:26:11.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.893+0000 7fd4c9800640 1 Processor -- start 2024-04-26T20:26:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.894+0000 7fd4c9800640 1 -- start start 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd4c41572b0 0x7fd4c40b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 0x7fd4c40d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4c40af8a0 0x7fd4c40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c415d2e0 con 0x7fd4c40af8a0 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c415d5c0 con 0x7fd4c41572b0 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.895+0000 7fd4c9800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c415d8a0 con 0x7fd4c40d0bd0 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4c40af8a0 0x7fd4c40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4c40af8a0 0x7fd4c40afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37638/0 (socket says 172.21.15.43:37638) 2024-04-26T20:26:11.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 -- 172.21.15.43:0/2676660101 learned_addr learned my addr 172.21.15.43:0/2676660101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c3e00640 1 --2- 172.21.15.43:0/2676660101 >> v2:172.21.15.164:3300/0 conn(0x7fd4c41572b0 0x7fd4c40b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c3400640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 0x7fd4c40d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 -- 172.21.15.43:0/2676660101 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 msgr2=0x7fd4c40d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 0x7fd4c40d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 -- 172.21.15.43:0/2676660101 >> v2:172.21.15.164:3300/0 conn(0x7fd4c41572b0 msgr2=0x7fd4c40b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/2676660101 >> v2:172.21.15.164:3300/0 conn(0x7fd4c41572b0 0x7fd4c40b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c8e00640 1 -- 172.21.15.43:0/2676660101 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c415db80 con 0x7fd4c40af8a0 2024-04-26T20:26:11.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.896+0000 7fd4c3400640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 0x7fd4c40d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.897+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c40af8a0 0x7fd4c40afd30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd4c406f210 tx=0x7fd4bc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a09336f9bfc3f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.898+0000 7fd4c2a00640 1 -- 172.21.15.43:0/2676660101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc036070 con 0x7fd4c40af8a0 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.898+0000 7fd4c2a00640 1 -- 172.21.15.43:0/2676660101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc031600 con 0x7fd4c40af8a0 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.898+0000 7fd4c2a00640 1 -- 172.21.15.43:0/2676660101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0319d0 con 0x7fd4c40af8a0 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.898+0000 7fd4c9800640 1 -- 172.21.15.43:0/2676660101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c40af8a0 msgr2=0x7fd4c40afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.898+0000 7fd4c9800640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c40af8a0 0x7fd4c40afd30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd4c406f210 tx=0x7fd4bc02a610 comp rx=0 tx=0).stop 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 -- 172.21.15.43:0/2676660101 shutdown_connections 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c40af8a0 0x7fd4c40afd30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 --2- 172.21.15.43:0/2676660101 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40d0bd0 0x7fd4c40d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 --2- 172.21.15.43:0/2676660101 >> v2:172.21.15.164:3300/0 conn(0x7fd4c41572b0 0x7fd4c40b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 -- 172.21.15.43:0/2676660101 >> 172.21.15.43:0/2676660101 conn(0x7fd4c40684e0 msgr2=0x7fd4c40688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 -- 172.21.15.43:0/2676660101 shutdown_connections 2024-04-26T20:26:11.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.899+0000 7fd4c9800640 1 -- 172.21.15.43:0/2676660101 wait complete. 2024-04-26T20:26:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.901+0000 7fd4c9800640 1 Processor -- start 2024-04-26T20:26:11.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.901+0000 7fd4c9800640 1 -- start start 2024-04-26T20:26:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40af8a0 0x7fd4c4157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4c40d0bd0 0x7fd4c41559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41609f0 con 0x7fd4c41572b0 2024-04-26T20:26:11.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4160870 con 0x7fd4c40d0bd0 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c9800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c4160b70 con 0x7fd4c40af8a0 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37650/0 (socket says 172.21.15.43:37650) 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c8e00640 1 -- 172.21.15.43:0/1526939549 learned_addr learned my addr 172.21.15.43:0/1526939549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.902+0000 7fd4c3e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40af8a0 0x7fd4c4157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c3400640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4c40d0bd0 0x7fd4c41559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 -- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40af8a0 msgr2=0x7fd4c4157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40af8a0 0x7fd4c4157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 -- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4c40d0bd0 msgr2=0x7fd4c41559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4c40d0bd0 0x7fd4c41559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 -- 172.21.15.43:0/1526939549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c4152400 con 0x7fd4c41572b0 2024-04-26T20:26:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c8e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc034040 tx=0x7fd4bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc036050 con 0x7fd4c41572b0 2024-04-26T20:26:11.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc042070 con 0x7fd4c41572b0 2024-04-26T20:26:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc009510 con 0x7fd4c41572b0 2024-04-26T20:26:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.903+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c4153e50 con 0x7fd4c41572b0 2024-04-26T20:26:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.904+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c41540f0 con 0x7fd4c41572b0 2024-04-26T20:26:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.905+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c40506c0 con 0x7fd4c41572b0 2024-04-26T20:26:11.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.906+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4bc007bd0 con 0x7fd4c41572b0 2024-04-26T20:26:11.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.907+0000 7fd4bbe00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 0x7fd4a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:11.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.907+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd4bc0acd60 con 0x7fd4c41572b0 2024-04-26T20:26:11.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.907+0000 7fd4c3e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 0x7fd4a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:11.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.908+0000 7fd4c3e00640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 0x7fd4a406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4006fd0 tx=0x7fd4b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:11.911+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4c4152400 con 0x7fd4c41572b0 2024-04-26T20:26:12.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.119+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd4c40cff80 con 0x7fd4c41572b0 2024-04-26T20:26:12.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.119+0000 7fd4bbe00640 1 -- 172.21.15.43:0/1526939549 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd4bc07ae40 con 0x7fd4c41572b0 2024-04-26T20:26:12.121 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.122+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 msgr2=0x7fd4a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.122+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 0x7fd4a406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4006fd0 tx=0x7fd4b4008040 comp rx=0 tx=0).stop 2024-04-26T20:26:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.122+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 msgr2=0x7fd4c4149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.122+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc034040 tx=0x7fd4bc004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:12.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 shutdown_connections 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd4a406c8f0 0x7fd4a406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4c41572b0 0x7fd4c4149fb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4c40d0bd0 0x7fd4c41559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 --2- 172.21.15.43:0/1526939549 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd4c40af8a0 0x7fd4c4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 >> 172.21.15.43:0/1526939549 conn(0x7fd4c40684e0 msgr2=0x7fd4c40ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 shutdown_connections 2024-04-26T20:26:12.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.123+0000 7fd4c9800640 1 -- 172.21.15.43:0/1526939549 wait complete. 2024-04-26T20:26:12.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:12.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.358+0000 7f8a07e00640 1 Processor -- start 2024-04-26T20:26:12.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.358+0000 7f8a07e00640 1 -- start start 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.360+0000 7f8a07e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a000b5250 0x7f8a000cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.360+0000 7f8a07e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0006eaf0 con 0x7f8a0015d5f0 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.360+0000 7f8a07e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0006edd0 con 0x7f8a000cf440 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.360+0000 7f8a07e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0006f420 con 0x7f8a000b5250 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a04a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a000b5250 0x7f8a000cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a05e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a0015d5f0 0x7f8a0015da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54322/0 (socket says 172.21.15.43:54322) 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a04a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a000b5250 0x7f8a000cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37656/0 (socket says 172.21.15.43:37656) 2024-04-26T20:26:12.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a05e00640 1 -- 172.21.15.43:0/2678701448 learned_addr learned my addr 172.21.15.43:0/2678701448 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:12.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a04a00640 1 -- 172.21.15.43:0/2678701448 >> v1:172.21.15.164:6789/0 conn(0x7f8a000cf440 legacy=0x7f8a000cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a04a00640 1 -- 172.21.15.43:0/2678701448 >> v1:172.21.15.43:6789/0 conn(0x7f8a0015d5f0 legacy=0x7f8a0015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.361+0000 7f8a04a00640 1 -- 172.21.15.43:0/2678701448 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0006f0b0 con 0x7f8a000b5250 2024-04-26T20:26:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.362+0000 7f8a04a00640 1 --2- 172.21.15.43:0/2678701448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000b5250 0x7f8a000cff30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0097a0 tx=0x7f89ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38fe8b64bb211c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.363+0000 7f89ffe00640 1 -- 172.21.15.43:0/2678701448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec037070 con 0x7f8a000b5250 2024-04-26T20:26:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.363+0000 7f89ffe00640 1 -- 172.21.15.43:0/2678701448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ec032440 con 0x7f8a000b5250 2024-04-26T20:26:12.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.363+0000 7f89ffe00640 1 -- 172.21.15.43:0/2678701448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec032810 con 0x7f8a000b5250 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 -- 172.21.15.43:0/2678701448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000b5250 msgr2=0x7f8a000cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 --2- 172.21.15.43:0/2678701448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000b5250 0x7f8a000cff30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0097a0 tx=0x7f89ec02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 -- 172.21.15.43:0/2678701448 shutdown_connections 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 --2- 172.21.15.43:0/2678701448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000b5250 0x7f8a000cff30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 -- 172.21.15.43:0/2678701448 >> 172.21.15.43:0/2678701448 conn(0x7f8a000b2900 msgr2=0x7f8a000b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 -- 172.21.15.43:0/2678701448 shutdown_connections 2024-04-26T20:26:12.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.364+0000 7f8a07e00640 1 -- 172.21.15.43:0/2678701448 wait complete. 2024-04-26T20:26:12.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.365+0000 7f8a07e00640 1 Processor -- start 2024-04-26T20:26:12.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.366+0000 7f8a07e00640 1 -- start start 2024-04-26T20:26:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a000b5250 0x7f8a001e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 0x7f8a000b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a000d0470 con 0x7f8a000cf440 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0006f8b0 con 0x7f8a0015d5f0 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a07e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0015e190 con 0x7f8a000b5250 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37666/0 (socket says 172.21.15.43:37666) 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 -- 172.21.15.43:0/3415909673 learned_addr learned my addr 172.21.15.43:0/3415909673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a05e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 0x7f8a000b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a05400640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a000b5250 0x7f8a001e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 -- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a000b5250 msgr2=0x7f8a001e65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a000b5250 0x7f8a001e65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 -- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 msgr2=0x7f8a000b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 0x7f8a000b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a04a00640 1 -- 172.21.15.43:0/3415909673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001428f0 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.367+0000 7f8a05e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 0x7f8a000b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f8a04a00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098c0 tx=0x7f89ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec041070 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ec03c870 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001e6cb0 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec037040 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.368+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0015eee0 con 0x7f8a000cf440 2024-04-26T20:26:12.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.369+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00069350 con 0x7f8a000cf440 2024-04-26T20:26:12.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.371+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89ec044070 con 0x7f8a000cf440 2024-04-26T20:26:12.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.372+0000 7f89fd600640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 0x7f89e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.372+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f89ec0ace20 con 0x7f8a000cf440 2024-04-26T20:26:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.372+0000 7f8a05400640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 0x7f89e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.373+0000 7f8a05400640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 0x7f89e006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89f4000f60 tx=0x7f89f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.376+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a001428f0 con 0x7f8a000cf440 2024-04-26T20:26:12.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.580+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8a000b1e00 con 0x7f8a000cf440 2024-04-26T20:26:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.581+0000 7f89fd600640 1 -- 172.21.15.43:0/3415909673 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f89ec07af80 con 0x7f8a000cf440 2024-04-26T20:26:12.583 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 msgr2=0x7f89e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 0x7f89e006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89f4000f60 tx=0x7f89f4009040 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 msgr2=0x7f8a001419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0098c0 tx=0x7f89ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 shutdown_connections 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f89e006c8f0 0x7f89e006edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a0015d5f0 0x7f8a000b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a000cf440 0x7f8a001419a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 --2- 172.21.15.43:0/3415909673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a000b5250 0x7f8a001e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 >> 172.21.15.43:0/3415909673 conn(0x7f8a000b2900 msgr2=0x7f8a000b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 shutdown_connections 2024-04-26T20:26:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.584+0000 7f8a07e00640 1 -- 172.21.15.43:0/3415909673 wait complete. 2024-04-26T20:26:12.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:26:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.815+0000 7fce11400640 1 Processor -- start 2024-04-26T20:26:12.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.815+0000 7fce11400640 1 -- start start 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.817+0000 7fce11400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce0c072620 0x7fce0c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.817+0000 7fce11400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fce0c0ce2b0 0x7fce0c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.817+0000 7fce11400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0c072a80 con 0x7fce0c0cd180 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.817+0000 7fce11400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c0ce6e0 con 0x7fce0c072620 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.817+0000 7fce11400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c0cd560 con 0x7fce0c0ce2b0 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fce0c0cd180 0x7fce0c15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54332/0 (socket says 172.21.15.43:54332) 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0be00640 1 -- 172.21.15.43:0/451828429 learned_addr learned my addr 172.21.15.43:0/451828429 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0b400640 1 --2- 172.21.15.43:0/451828429 >> v2:172.21.15.43:3300/0 conn(0x7fce0c072620 0x7fce0c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce10a00640 1 --2- 172.21.15.43:0/451828429 >> v2:172.21.15.164:3300/0 conn(0x7fce0c0ce2b0 0x7fce0c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0aa00640 1 -- 172.21.15.43:0/451828429 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3335887832 0 0) 0x7fcdfc001110 con 0x7fce0c0cd180 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0aa00640 1 -- 172.21.15.43:0/451828429 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdf0003520 con 0x7fce0c0cd180 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0b400640 1 -- 172.21.15.43:0/451828429 >> v2:172.21.15.164:3300/0 conn(0x7fce0c0ce2b0 msgr2=0x7fce0c06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0b400640 1 --2- 172.21.15.43:0/451828429 >> v2:172.21.15.164:3300/0 conn(0x7fce0c0ce2b0 0x7fce0c06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0b400640 1 -- 172.21.15.43:0/451828429 >> v1:172.21.15.43:6789/0 conn(0x7fce0c0cd180 legacy=0x7fce0c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.818+0000 7fce0b400640 1 -- 172.21.15.43:0/451828429 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c072d00 con 0x7fce0c072620 2024-04-26T20:26:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.819+0000 7fce0b400640 1 --2- 172.21.15.43:0/451828429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c0cdd70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0b39f0 tx=0x7fcdf402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbba866956f3827 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce0aa00640 1 -- 172.21.15.43:0/451828429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4007740 con 0x7fce0c072620 2024-04-26T20:26:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce0aa00640 1 -- 172.21.15.43:0/451828429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf40078e0 con 0x7fce0c072620 2024-04-26T20:26:12.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce0aa00640 1 -- 172.21.15.43:0/451828429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4007cb0 con 0x7fce0c072620 2024-04-26T20:26:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 -- 172.21.15.43:0/451828429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 msgr2=0x7fce0c0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 --2- 172.21.15.43:0/451828429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c0cdd70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0b39f0 tx=0x7fcdf402a680 comp rx=0 tx=0).stop 2024-04-26T20:26:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 -- 172.21.15.43:0/451828429 shutdown_connections 2024-04-26T20:26:12.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 --2- 172.21.15.43:0/451828429 >> v2:172.21.15.164:3300/0 conn(0x7fce0c0ce2b0 0x7fce0c06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 --2- 172.21.15.43:0/451828429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c0cdd70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.820+0000 7fce11400640 1 -- 172.21.15.43:0/451828429 >> 172.21.15.43:0/451828429 conn(0x7fce0c068090 msgr2=0x7fce0c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:12.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.821+0000 7fce11400640 1 -- 172.21.15.43:0/451828429 shutdown_connections 2024-04-26T20:26:12.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.821+0000 7fce11400640 1 -- 172.21.15.43:0/451828429 wait complete. 2024-04-26T20:26:12.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.822+0000 7fce11400640 1 Processor -- start 2024-04-26T20:26:12.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.823+0000 7fce11400640 1 -- start start 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.823+0000 7fce11400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce11400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0c0cd180 0x7fce0c140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce11400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce0c0ce2b0 0x7fce0c1449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce11400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0735a0 con 0x7fce0c072620 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce11400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0732c0 con 0x7fce0c0ce2b0 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce11400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c073da0 con 0x7fce0c0cd180 2024-04-26T20:26:12.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37692/0 (socket says 172.21.15.43:37692) 2024-04-26T20:26:12.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 -- 172.21.15.43:0/2347084067 learned_addr learned my addr 172.21.15.43:0/2347084067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce10a00640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce0c0ce2b0 0x7fce0c1449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 -- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0c0cd180 msgr2=0x7fce0c140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0c0cd180 0x7fce0c140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 -- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce0c0ce2b0 msgr2=0x7fce0c1449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce0c0ce2b0 0x7fce0c1449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.824+0000 7fce0be00640 1 -- 172.21.15.43:0/2347084067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c13efc0 con 0x7fce0c072620 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce0be00640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c760 tx=0x7fcdfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc017070 con 0x7fce0c072620 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc005ad0 con 0x7fce0c072620 2024-04-26T20:26:12.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c13f2a0 con 0x7fce0c072620 2024-04-26T20:26:12.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc007e40 con 0x7fce0c072620 2024-04-26T20:26:12.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.825+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c13f860 con 0x7fce0c072620 2024-04-26T20:26:12.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.826+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c069350 con 0x7fce0c072620 2024-04-26T20:26:12.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.827+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdfc01a070 con 0x7fce0c072620 2024-04-26T20:26:12.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.828+0000 7fce03e00640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.828+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcdfc08b2d0 con 0x7fce0c072620 2024-04-26T20:26:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.829+0000 7fce0b400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:12.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.829+0000 7fce0b400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 0x7fcde806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034000 tx=0x7fcdf402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:12.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:12.833+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce0c072a80 con 0x7fce0c072620 2024-04-26T20:26:13.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.059+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce0c0720e0 con 0x7fce0c072620 2024-04-26T20:26:13.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.060+0000 7fce03e00640 1 -- 172.21.15.43:0/2347084067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fcdfc059320 con 0x7fce0c072620 2024-04-26T20:26:13.061 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:26:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.062+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 msgr2=0x7fcde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.062+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 0x7fcde806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4034000 tx=0x7fcdf402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.062+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 msgr2=0x7fce0c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.062+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c760 tx=0x7fcdfc00cc20 comp rx=0 tx=0).stop 2024-04-26T20:26:13.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 shutdown_connections 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce0c0cd180 0x7fce0c140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce0c0ce2b0 0x7fce0c1449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 --2- 172.21.15.43:0/2347084067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce0c072620 0x7fce0c15c910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 >> 172.21.15.43:0/2347084067 conn(0x7fce0c068090 msgr2=0x7fce0c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 shutdown_connections 2024-04-26T20:26:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.063+0000 7fce11400640 1 -- 172.21.15.43:0/2347084067 wait complete. 2024-04-26T20:26:13.084 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:26:13.084 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:26:13.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.377+0000 7ff1c8e00640 1 Processor -- start 2024-04-26T20:26:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.377+0000 7ff1c8e00640 1 -- start start 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.378+0000 7ff1c8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1c40d4df0 0x7ff1c40d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.378+0000 7ff1c8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1c40b4580 0x7ff1c40b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.378+0000 7ff1c8e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c40d3c60 con 0x7ff1c415f3c0 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.378+0000 7ff1c8e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c40b4aa0 con 0x7ff1c40d4df0 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.378+0000 7ff1c8e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c40d3f10 con 0x7ff1c40b4580 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1c40d4df0 0x7ff1c40d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1c40d4df0 0x7ff1c40d3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37704/0 (socket says 172.21.15.43:37704) 2024-04-26T20:26:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 -- 172.21.15.43:0/569680308 learned_addr learned my addr 172.21.15.43:0/569680308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 -- 172.21.15.43:0/569680308 >> v2:172.21.15.164:3300/0 conn(0x7ff1c40b4580 msgr2=0x7ff1c40b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c2800640 1 --2- 172.21.15.43:0/569680308 >> v2:172.21.15.164:3300/0 conn(0x7ff1c40b4580 0x7ff1c40b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 --2- 172.21.15.43:0/569680308 >> v2:172.21.15.164:3300/0 conn(0x7ff1c40b4580 0x7ff1c40b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 -- 172.21.15.43:0/569680308 >> v1:172.21.15.43:6789/0 conn(0x7ff1c415f3c0 legacy=0x7ff1c40d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c1e00640 1 -- 172.21.15.43:0/569680308 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c40b4cf0 con 0x7ff1c40d4df0 2024-04-26T20:26:13.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.379+0000 7ff1c0a00640 1 -- 172.21.15.43:0/569680308 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273724576 0 0) 0x7ff1c40d3c60 con 0x7ff1c415f3c0 2024-04-26T20:26:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.380+0000 7ff1c1e00640 1 --2- 172.21.15.43:0/569680308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40d4df0 0x7ff1c40d3720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c8ba0 tx=0x7ff1ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cca917fdb04eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c0a00640 1 -- 172.21.15.43:0/569680308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac037070 con 0x7ff1c40d4df0 2024-04-26T20:26:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c0a00640 1 -- 172.21.15.43:0/569680308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac032440 con 0x7ff1c40d4df0 2024-04-26T20:26:13.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c0a00640 1 -- 172.21.15.43:0/569680308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac032810 con 0x7ff1c40d4df0 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 -- 172.21.15.43:0/569680308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40d4df0 msgr2=0x7ff1c40d3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/569680308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40d4df0 0x7ff1c40d3720 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c8ba0 tx=0x7ff1ac02a270 comp rx=0 tx=0).stop 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 -- 172.21.15.43:0/569680308 shutdown_connections 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/569680308 >> v2:172.21.15.164:3300/0 conn(0x7ff1c40b4580 0x7ff1c40b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/569680308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40d4df0 0x7ff1c40d3720 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 -- 172.21.15.43:0/569680308 >> 172.21.15.43:0/569680308 conn(0x7ff1c408ebd0 msgr2=0x7ff1c409e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 -- 172.21.15.43:0/569680308 shutdown_connections 2024-04-26T20:26:13.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.381+0000 7ff1c8e00640 1 -- 172.21.15.43:0/569680308 wait complete. 2024-04-26T20:26:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.383+0000 7ff1c8e00640 1 Processor -- start 2024-04-26T20:26:13.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.383+0000 7ff1c8e00640 1 -- start start 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 0x7ff1c40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 0x7ff1c40b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c40b5d10 con 0x7ff1c40b4580 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c40b52b0 con 0x7ff1c415f3c0 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c8e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c40b5e90 con 0x7ff1c40d4df0 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37706/0 (socket says 172.21.15.43:37706) 2024-04-26T20:26:13.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.384+0000 7ff1c1e00640 1 -- 172.21.15.43:0/4120807328 learned_addr learned my addr 172.21.15.43:0/4120807328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:13.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c2800640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 0x7ff1c40b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1400640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 0x7ff1c40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 -- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 msgr2=0x7ff1c40c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 0x7ff1c40c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 -- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 msgr2=0x7ff1c40b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 0x7ff1c40b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 -- 172.21.15.43:0/4120807328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c40b9700 con 0x7ff1c40b4580 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c2800640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 0x7ff1c40b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1400640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 0x7ff1c40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1c1e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.385+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac037040 con 0x7ff1c40b4580 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.386+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac042070 con 0x7ff1c40b4580 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.386+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac030bb0 con 0x7ff1c40b4580 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.386+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c40c96c0 con 0x7ff1c40b4580 2024-04-26T20:26:13.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.386+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c40c9c80 con 0x7ff1c40b4580 2024-04-26T20:26:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.387+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c40ca740 con 0x7ff1c40b4580 2024-04-26T20:26:13.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.388+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1ac047460 con 0x7ff1c40b4580 2024-04-26T20:26:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.389+0000 7ff1ba000640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 0x7ff1a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.389+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff1ac0acb80 con 0x7ff1c40b4580 2024-04-26T20:26:13.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.389+0000 7ff1c1400640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 0x7ff1a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.390+0000 7ff1c1400640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 0x7ff1a006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007fd0 tx=0x7ff1b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.393+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1c40ca740 con 0x7ff1c40b4580 2024-04-26T20:26:13.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.603+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c40cf550 con 0x7ff1c40b4580 2024-04-26T20:26:13.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.606+0000 7ff1ba000640 1 -- 172.21.15.43:0/4120807328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7ff1c40b9700 con 0x7ff1c40b4580 2024-04-26T20:26:13.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:13.609 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 msgr2=0x7ff1a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 0x7ff1a006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007fd0 tx=0x7ff1b4009040 comp rx=0 tx=0).stop 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 msgr2=0x7ff1c40b91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000c00 tx=0x7ff1ac0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 shutdown_connections 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff1a006c920 0x7ff1a006ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff1c40d4df0 0x7ff1c40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1c415f3c0 0x7ff1c40b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 --2- 172.21.15.43:0/4120807328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1c40b4580 0x7ff1c40b91c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.609+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 >> 172.21.15.43:0/4120807328 conn(0x7ff1c408ebd0 msgr2=0x7ff1c40d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.610+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 shutdown_connections 2024-04-26T20:26:13.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.610+0000 7ff1c8e00640 1 -- 172.21.15.43:0/4120807328 wait complete. 2024-04-26T20:26:13.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:13.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:13.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.848+0000 7fb955400640 1 Processor -- start 2024-04-26T20:26:13.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.848+0000 7fb955400640 1 -- start start 2024-04-26T20:26:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb955400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb95015f1e0 0x7fb95006e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb955400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb950071ee0 0x7fb950072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb955400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95006e7b0 con 0x7fb9500b57e0 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb955400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb95015f5c0 con 0x7fb950071ee0 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb955400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9500729a0 con 0x7fb95015f1e0 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb947e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb950071ee0 0x7fb950072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb94f200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9500b57e0 0x7fb9500b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54354/0 (socket says 172.21.15.43:54354) 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb94f200640 1 -- 172.21.15.43:0/4179482623 learned_addr learned my addr 172.21.15.43:0/4179482623 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.850+0000 7fb94e800640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb95015f1e0 0x7fb95006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.851+0000 7fb947e00640 1 -- 172.21.15.43:0/4179482623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb95015f1e0 msgr2=0x7fb95006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.851+0000 7fb947e00640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb95015f1e0 0x7fb95006e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.851+0000 7fb947e00640 1 -- 172.21.15.43:0/4179482623 >> v1:172.21.15.43:6789/0 conn(0x7fb9500b57e0 legacy=0x7fb9500b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.851+0000 7fb947e00640 1 -- 172.21.15.43:0/4179482623 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb950072c80 con 0x7fb950071ee0 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.851+0000 7fb94de00640 1 -- 172.21.15.43:0/4179482623 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714121534 0 0) 0x7fb95006e7b0 con 0x7fb9500b57e0 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.852+0000 7fb947e00640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950072340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097a0 tx=0x7fb93802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df2054ac0abab85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.852+0000 7fb94de00640 1 -- 172.21.15.43:0/4179482623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938037070 con 0x7fb950071ee0 2024-04-26T20:26:13.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.852+0000 7fb94de00640 1 -- 172.21.15.43:0/4179482623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb938032440 con 0x7fb950071ee0 2024-04-26T20:26:13.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.852+0000 7fb94de00640 1 -- 172.21.15.43:0/4179482623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938032810 con 0x7fb950071ee0 2024-04-26T20:26:13.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 -- 172.21.15.43:0/4179482623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 msgr2=0x7fb950072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950072340 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097a0 tx=0x7fb93802a180 comp rx=0 tx=0).stop 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 -- 172.21.15.43:0/4179482623 shutdown_connections 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950072340 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 --2- 172.21.15.43:0/4179482623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb95015f1e0 0x7fb95006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 -- 172.21.15.43:0/4179482623 >> 172.21.15.43:0/4179482623 conn(0x7fb9500b2900 msgr2=0x7fb9500b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 -- 172.21.15.43:0/4179482623 shutdown_connections 2024-04-26T20:26:13.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.853+0000 7fb955400640 1 -- 172.21.15.43:0/4179482623 wait complete. 2024-04-26T20:26:13.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.854+0000 7fb955400640 1 Processor -- start 2024-04-26T20:26:13.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.855+0000 7fb955400640 1 -- start start 2024-04-26T20:26:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9500b57e0 0x7fb95015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 0x7fb95015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb95015f9d0 con 0x7fb950071ee0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9500b42c0 con 0x7fb95015f1e0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb955400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9500b3f70 con 0x7fb9500b57e0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37730/0 (socket says 172.21.15.43:37730) 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 -- 172.21.15.43:0/2726391631 learned_addr learned my addr 172.21.15.43:0/2726391631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb947e00640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9500b57e0 0x7fb95015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 -- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9500b57e0 msgr2=0x7fb95015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9500b57e0 0x7fb95015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94f200640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 0x7fb95015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 -- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 msgr2=0x7fb95015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.856+0000 7fb94e800640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 0x7fb95015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb94e800640 1 -- 172.21.15.43:0/2726391631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95015d3e0 con 0x7fb950071ee0 2024-04-26T20:26:13.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb94f200640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 0x7fb95015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb94e800640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d800 tx=0x7fb94000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000b6c0 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940019070 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95015d6c0 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000ba80 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.857+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95015dc80 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.859+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb950069350 con 0x7fb950071ee0 2024-04-26T20:26:13.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.859+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb94000bc20 con 0x7fb950071ee0 2024-04-26T20:26:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.860+0000 7fb946a00640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:13.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.860+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb94008c420 con 0x7fb950071ee0 2024-04-26T20:26:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.861+0000 7fb947e00640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:13.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.861+0000 7fb947e00640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb938034000 tx=0x7fb93802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:13.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:13.865+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb94001c360 con 0x7fb950071ee0 2024-04-26T20:26:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.075+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9500b26e0 con 0x7fb950071ee0 2024-04-26T20:26:14.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.076+0000 7fb946a00640 1 -- 172.21.15.43:0/2726391631 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fb94005a470 con 0x7fb950071ee0 2024-04-26T20:26:14.078 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 msgr2=0x7fb92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb938034000 tx=0x7fb93802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 msgr2=0x7fb950143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d800 tx=0x7fb94000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 shutdown_connections 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9500b57e0 0x7fb95015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb95015f1e0 0x7fb95015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 --2- 172.21.15.43:0/2726391631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb950071ee0 0x7fb950143de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.079+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 >> 172.21.15.43:0/2726391631 conn(0x7fb9500b2900 msgr2=0x7fb95006e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.080+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 shutdown_connections 2024-04-26T20:26:14.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.080+0000 7fb955400640 1 -- 172.21.15.43:0/2726391631 wait complete. 2024-04-26T20:26:14.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.314+0000 7f2218e00640 1 Processor -- start 2024-04-26T20:26:14.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.315+0000 7f2218e00640 1 -- start start 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2218e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22140d3830 0x7f22140d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2218e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140d2d70 con 0x7f221415f180 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2218e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140d06e0 con 0x7f22140b3cb0 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2218e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f221415f560 con 0x7f22140d3830 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2213400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f221415f180 0x7f22140d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54364/0 (socket says 172.21.15.43:54364) 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2213400640 1 -- 172.21.15.43:0/1823554353 learned_addr learned my addr 172.21.15.43:0/1823554353 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:14.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.316+0000 7f2212a00640 1 --2- 172.21.15.43:0/1823554353 >> v2:172.21.15.43:3300/0 conn(0x7f22140d3830 0x7f22140d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.317+0000 7f2212000640 1 -- 172.21.15.43:0/1823554353 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202792714 0 0) 0x7f22140d2d70 con 0x7f221415f180 2024-04-26T20:26:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.317+0000 7f2212000640 1 -- 172.21.15.43:0/1823554353 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21f4003520 con 0x7f221415f180 2024-04-26T20:26:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.317+0000 7f2212a00640 1 -- 172.21.15.43:0/1823554353 >> v1:172.21.15.164:6789/0 conn(0x7f22140b3cb0 legacy=0x7f22140b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.317+0000 7f2212a00640 1 -- 172.21.15.43:0/1823554353 >> v1:172.21.15.43:6789/0 conn(0x7f221415f180 legacy=0x7f22140d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.317+0000 7f2212a00640 1 -- 172.21.15.43:0/1823554353 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140d09c0 con 0x7f22140d3830 2024-04-26T20:26:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.318+0000 7f2212a00640 1 --2- 172.21.15.43:0/1823554353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22140d3830 0x7f22140d56f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1753eeba316ccafe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.318+0000 7f2212000640 1 -- 172.21.15.43:0/1823554353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc007740 con 0x7f22140d3830 2024-04-26T20:26:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.318+0000 7f2212000640 1 -- 172.21.15.43:0/1823554353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc0078e0 con 0x7f22140d3830 2024-04-26T20:26:14.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.318+0000 7f2212000640 1 -- 172.21.15.43:0/1823554353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc007cb0 con 0x7f22140d3830 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 -- 172.21.15.43:0/1823554353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22140d3830 msgr2=0x7f22140d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 --2- 172.21.15.43:0/1823554353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22140d3830 0x7f22140d56f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 -- 172.21.15.43:0/1823554353 shutdown_connections 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 --2- 172.21.15.43:0/1823554353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22140d3830 0x7f22140d56f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 -- 172.21.15.43:0/1823554353 >> 172.21.15.43:0/1823554353 conn(0x7f22140d4bd0 msgr2=0x7f22140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 -- 172.21.15.43:0/1823554353 shutdown_connections 2024-04-26T20:26:14.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.319+0000 7f2218e00640 1 -- 172.21.15.43:0/1823554353 wait complete. 2024-04-26T20:26:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.321+0000 7f2218e00640 1 Processor -- start 2024-04-26T20:26:14.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.321+0000 7f2218e00640 1 -- start start 2024-04-26T20:26:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22140b3cb0 0x7f22140b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140d3830 0x7f22140c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d11c0 con 0x7f221415f180 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d2a40 con 0x7f22140b3cb0 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2218e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22140ba6a0 con 0x7f22140d3830 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2213e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2213e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37744/0 (socket says 172.21.15.43:37744) 2024-04-26T20:26:14.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2213e00640 1 -- 172.21.15.43:0/2420346822 learned_addr learned my addr 172.21.15.43:0/2420346822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.322+0000 7f2212a00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140d3830 0x7f22140c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213400640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22140b3cb0 0x7f22140b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 -- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140d3830 msgr2=0x7f22140c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140d3830 0x7f22140c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 -- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22140b3cb0 msgr2=0x7f22140b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22140b3cb0 0x7f22140b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 -- 172.21.15.43:0/2420346822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22140b90c0 con 0x7f221415f180 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2213e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c3e0 tx=0x7f220c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c011790 con 0x7f221415f180 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c017070 con 0x7f221415f180 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22140b93a0 con 0x7f221415f180 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.323+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c011b70 con 0x7f221415f180 2024-04-26T20:26:14.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.324+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22140b9960 con 0x7f221415f180 2024-04-26T20:26:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.325+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22140506c0 con 0x7f221415f180 2024-04-26T20:26:14.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.326+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f220c011d10 con 0x7f221415f180 2024-04-26T20:26:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.327+0000 7f220b400640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 0x7f21f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.327+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f220c08bcd0 con 0x7f221415f180 2024-04-26T20:26:14.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.327+0000 7f2213400640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 0x7f21f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.328+0000 7f2213400640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 0x7f21f006ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2204005880 tx=0x7f2204009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.331+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f220c059d20 con 0x7f221415f180 2024-04-26T20:26:14.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.536+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22140baa60 con 0x7f221415f180 2024-04-26T20:26:14.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.537+0000 7f220b400640 1 -- 172.21.15.43:0/2420346822 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f220c059620 con 0x7f221415f180 2024-04-26T20:26:14.538 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:14.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.539+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 msgr2=0x7f21f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.539+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 0x7f21f006ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2204005880 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-04-26T20:26:14.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.539+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 msgr2=0x7f22140ba160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.539+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f220c00c3e0 tx=0x7f220c00c8a0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 shutdown_connections 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21f006c920 0x7f21f006ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140d3830 0x7f22140c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221415f180 0x7f22140ba160 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 --2- 172.21.15.43:0/2420346822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22140b3cb0 0x7f22140b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 >> 172.21.15.43:0/2420346822 conn(0x7f22140d4bd0 msgr2=0x7f22140d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 shutdown_connections 2024-04-26T20:26:14.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.540+0000 7f2218e00640 1 -- 172.21.15.43:0/2420346822 wait complete. 2024-04-26T20:26:14.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:14.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.776+0000 7f77ba600640 1 Processor -- start 2024-04-26T20:26:14.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.777+0000 7f77ba600640 1 -- start start 2024-04-26T20:26:14.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77ba600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77b4073740 0x7f77b4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77ba600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b40b48f0 con 0x7f77b4159870 2024-04-26T20:26:14.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77ba600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b40b4bd0 con 0x7f77b40b3c00 2024-04-26T20:26:14.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77ba600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f77b4159c50 con 0x7f77b4073740 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77b2a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77b4073740 0x7f77b4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77b3400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77b4159870 0x7f77b406efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54380/0 (socket says 172.21.15.43:54380) 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77b2a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77b4073740 0x7f77b4073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37752/0 (socket says 172.21.15.43:37752) 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.779+0000 7f77b3400640 1 -- 172.21.15.43:0/1829163707 learned_addr learned my addr 172.21.15.43:0/1829163707 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.780+0000 7f77b2a00640 1 -- 172.21.15.43:0/1829163707 >> v1:172.21.15.164:6789/0 conn(0x7f77b40b3c00 legacy=0x7f77b40b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:14.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.780+0000 7f77b2000640 1 -- 172.21.15.43:0/1829163707 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772680822 0 0) 0x7f77b40b48f0 con 0x7f77b4159870 2024-04-26T20:26:14.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.780+0000 7f77b2a00640 1 -- 172.21.15.43:0/1829163707 >> v1:172.21.15.43:6789/0 conn(0x7f77b4159870 legacy=0x7f77b406efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.780+0000 7f77b2a00640 1 -- 172.21.15.43:0/1829163707 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b406e4c0 con 0x7f77b4073740 2024-04-26T20:26:14.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.781+0000 7f77b2a00640 1 --2- 172.21.15.43:0/1829163707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b4073bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77b4159dd0 tx=0x7f77a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7534d831ec13a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.781+0000 7f77b2000640 1 -- 172.21.15.43:0/1829163707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037070 con 0x7f77b4073740 2024-04-26T20:26:14.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.781+0000 7f77b2000640 1 -- 172.21.15.43:0/1829163707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a0032440 con 0x7f77b4073740 2024-04-26T20:26:14.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.781+0000 7f77b2000640 1 -- 172.21.15.43:0/1829163707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032810 con 0x7f77b4073740 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 -- 172.21.15.43:0/1829163707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 msgr2=0x7f77b4073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 --2- 172.21.15.43:0/1829163707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b4073bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f77b4159dd0 tx=0x7f77a002a210 comp rx=0 tx=0).stop 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 -- 172.21.15.43:0/1829163707 shutdown_connections 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 --2- 172.21.15.43:0/1829163707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b4073bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 -- 172.21.15.43:0/1829163707 >> 172.21.15.43:0/1829163707 conn(0x7f77b40b2900 msgr2=0x7f77b40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 -- 172.21.15.43:0/1829163707 shutdown_connections 2024-04-26T20:26:14.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.782+0000 7f77ba600640 1 -- 172.21.15.43:0/1829163707 wait complete. 2024-04-26T20:26:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.784+0000 7f77ba600640 1 Processor -- start 2024-04-26T20:26:14.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.784+0000 7f77ba600640 1 -- start start 2024-04-26T20:26:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77b40b3c00 0x7f77b415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 0x7f77b415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77b40720e0 con 0x7f77b4073740 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77b4071f60 con 0x7f77b4159870 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77ba600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77b4072260 con 0x7f77b40b3c00 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37758/0 (socket says 172.21.15.43:37758) 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 -- 172.21.15.43:0/3573382105 learned_addr learned my addr 172.21.15.43:0/3573382105 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b2a00640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77b40b3c00 0x7f77b415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3e00640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 0x7f77b415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 -- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77b40b3c00 msgr2=0x7f77b415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77b40b3c00 0x7f77b415ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 -- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 msgr2=0x7f77b415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 0x7f77b415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3400640 1 -- 172.21.15.43:0/3573382105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b4155f10 con 0x7f77b4073740 2024-04-26T20:26:14.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.785+0000 7f77b3e00640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 0x7f77b415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77b3400640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c3f0 tx=0x7f77a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a40074e0 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a40047e0 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004c10 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b41561f0 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.786+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b41567b0 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.787+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b4069350 con 0x7f77b4073740 2024-04-26T20:26:14.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.788+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77a4011070 con 0x7f77b4073740 2024-04-26T20:26:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.789+0000 7f77ab400640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 0x7f779406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:14.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.789+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f77a408bbf0 con 0x7f77b4073740 2024-04-26T20:26:14.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.790+0000 7f77b2a00640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 0x7f779406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:14.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.790+0000 7f77b2a00640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 0x7f779406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:14.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:14.794+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77a401a2b0 con 0x7f77b4073740 2024-04-26T20:26:15.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.001+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77b40b24b0 con 0x7f77b4073740 2024-04-26T20:26:15.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.002+0000 7f77ab400640 1 -- 172.21.15.43:0/3573382105 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f77a4059bb0 con 0x7f77b4073740 2024-04-26T20:26:15.004 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:15.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 msgr2=0x7f779406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 0x7f779406edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 msgr2=0x7f77b41558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c3f0 tx=0x7f77a400c8b0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 shutdown_connections 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f779406c8f0 0x7f779406edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77b40b3c00 0x7f77b415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4159870 0x7f77b415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 --2- 172.21.15.43:0/3573382105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b4073740 0x7f77b41558c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 >> 172.21.15.43:0/3573382105 conn(0x7f77b40b2900 msgr2=0x7f77b415e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 shutdown_connections 2024-04-26T20:26:15.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.005+0000 7f77ba600640 1 -- 172.21.15.43:0/3573382105 wait complete. 2024-04-26T20:26:15.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:15.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.240+0000 7f83a1400640 1 Processor -- start 2024-04-26T20:26:15.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.241+0000 7f83a1400640 1 -- start start 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.242+0000 7f83a1400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f839c15a7e0 0x7f839c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.242+0000 7f83a1400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 0x7f839c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.242+0000 7f83a1400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c0b3900 con 0x7f839c0b5410 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.242+0000 7f83a1400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f839c15abc0 con 0x7f839c15a7e0 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.242+0000 7f83a1400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0b3b70 con 0x7f839c15d9c0 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839be00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f839c15a7e0 0x7f839c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f8393e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f839c0b5410 0x7f839c15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54384/0 (socket says 172.21.15.43:54384) 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f8393e00640 1 -- 172.21.15.43:0/2363421597 learned_addr learned my addr 172.21.15.43:0/2363421597 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f83a0a00640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 0x7f839c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839be00640 1 -- 172.21.15.43:0/2363421597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 msgr2=0x7f839c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839be00640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 0x7f839c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839be00640 1 -- 172.21.15.43:0/2363421597 >> v1:172.21.15.43:6789/0 conn(0x7f839c0b5410 legacy=0x7f839c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839be00640 1 -- 172.21.15.43:0/2363421597 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b3e50 con 0x7f839c15a7e0 2024-04-26T20:26:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f839b400640 1 -- 172.21.15.43:0/2363421597 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756278224 0 0) 0x7f839c0b3900 con 0x7f839c0b5410 2024-04-26T20:26:15.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.243+0000 7f83a0a00640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 0x7f839c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f839be00640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15a7e0 0x7f839c0b4650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838c0097a0 tx=0x7f838c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd102ac12c96b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f839b400640 1 -- 172.21.15.43:0/2363421597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c037070 con 0x7f839c15a7e0 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f839b400640 1 -- 172.21.15.43:0/2363421597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c032440 con 0x7f839c15a7e0 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f839b400640 1 -- 172.21.15.43:0/2363421597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c032810 con 0x7f839c15a7e0 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f83a1400640 1 -- 172.21.15.43:0/2363421597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15a7e0 msgr2=0x7f839c0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.245+0000 7f83a1400640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15a7e0 0x7f839c0b4650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f838c0097a0 tx=0x7f838c02a180 comp rx=0 tx=0).stop 2024-04-26T20:26:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 -- 172.21.15.43:0/2363421597 shutdown_connections 2024-04-26T20:26:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15d9c0 0x7f839c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 --2- 172.21.15.43:0/2363421597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15a7e0 0x7f839c0b4650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 -- 172.21.15.43:0/2363421597 >> 172.21.15.43:0/2363421597 conn(0x7f839c068090 msgr2=0x7f839c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 -- 172.21.15.43:0/2363421597 shutdown_connections 2024-04-26T20:26:15.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.246+0000 7f83a1400640 1 -- 172.21.15.43:0/2363421597 wait complete. 2024-04-26T20:26:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.247+0000 7f83a1400640 1 Processor -- start 2024-04-26T20:26:15.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 -- start start 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 0x7f839c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 0x7f839c0b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0b4b90 con 0x7f839c15d9c0 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f839c15e010 con 0x7f839c0b5410 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.248+0000 7f83a1400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f839c15e2e0 con 0x7f839c15a7e0 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37772/0 (socket says 172.21.15.43:37772) 2024-04-26T20:26:15.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 -- 172.21.15.43:0/2087840368 learned_addr learned my addr 172.21.15.43:0/2087840368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f8393e00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 0x7f839c0b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f839be00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 0x7f839c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 -- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 msgr2=0x7f839c0b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 0x7f839c0b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 -- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 msgr2=0x7f839c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 0x7f839c153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f83a0a00640 1 -- 172.21.15.43:0/2087840368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1e6380 con 0x7f839c15d9c0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f8393e00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 0x7f839c0b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.249+0000 7f839be00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 0x7f839c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f83a0a00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f839400cd80 tx=0x7f839400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400f680 con 0x7f839c15d9c0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839400f840 con 0x7f839c15d9c0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400fc30 con 0x7f839c15d9c0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c1e6660 con 0x7f839c15d9c0 2024-04-26T20:26:15.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.250+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1e6c20 con 0x7f839c15d9c0 2024-04-26T20:26:15.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.251+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c0506c0 con 0x7f839c15d9c0 2024-04-26T20:26:15.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.252+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83940213f0 con 0x7f839c15d9c0 2024-04-26T20:26:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.253+0000 7f8398c00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 0x7f837806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.253+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f839408cb60 con 0x7f839c15d9c0 2024-04-26T20:26:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.254+0000 7f839be00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 0x7f837806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.254+0000 7f839be00640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 0x7f837806ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.258+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8394014030 con 0x7f839c15d9c0 2024-04-26T20:26:15.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.460+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f839c06f270 con 0x7f839c15d9c0 2024-04-26T20:26:15.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.461+0000 7f8398c00640 1 -- 172.21.15.43:0/2087840368 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f839405a6e0 con 0x7f839c15d9c0 2024-04-26T20:26:15.462 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 msgr2=0x7f837806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 0x7f837806ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 msgr2=0x7f839c1f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f839400cd80 tx=0x7f839400a650 comp rx=0 tx=0).stop 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 shutdown_connections 2024-04-26T20:26:15.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f837806c940 0x7f837806ee00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f839c15a7e0 0x7f839c0b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f839c15d9c0 0x7f839c1f2840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 --2- 172.21.15.43:0/2087840368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f839c0b5410 0x7f839c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 >> 172.21.15.43:0/2087840368 conn(0x7f839c068090 msgr2=0x7f839c15c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.463+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 shutdown_connections 2024-04-26T20:26:15.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.464+0000 7f83a1400640 1 -- 172.21.15.43:0/2087840368 wait complete. 2024-04-26T20:26:15.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:26:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.689+0000 7fae70a00640 1 Processor -- start 2024-04-26T20:26:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.689+0000 7fae70a00640 1 -- start start 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae70a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae6c162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae70a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fae6c0af720 0x7fae6c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae70a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae6c145710 con 0x7fae6c145330 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae70a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fae6c163280 con 0x7fae6c0af720 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae70a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c073dc0 con 0x7fae6c1628e0 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae6b400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae6c145330 0x7fae6c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54396/0 (socket says 172.21.15.43:54396) 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.691+0000 7fae6b400640 1 -- 172.21.15.43:0/3925665497 learned_addr learned my addr 172.21.15.43:0/3925665497 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6be00640 1 --2- 172.21.15.43:0/3925665497 >> v2:172.21.15.164:3300/0 conn(0x7fae6c0af720 0x7fae6c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6aa00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae6c162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557463433 0 0) 0x7fae6c145710 con 0x7fae6c145330 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae50003520 con 0x7fae6c145330 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 154534441 0 0) 0x7fae50003520 con 0x7fae6c145330 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 msgr2=0x7fae6c162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae6c162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 >> v2:172.21.15.164:3300/0 conn(0x7fae6c0af720 msgr2=0x7fae6c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 --2- 172.21.15.43:0/3925665497 >> v2:172.21.15.164:3300/0 conn(0x7fae6c0af720 0x7fae6c0afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0ae960 con 0x7fae6c145330 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fae5c001670 con 0x7fae6c145330 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 >> v1:172.21.15.43:6789/0 conn(0x7fae6c145330 legacy=0x7fae6c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae500039b0 0x7fae50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae50004d40 0x7fae500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c073dc0 con 0x7fae50004d40 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae5000b9c0 con 0x7fae500039b0 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.692+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fae5000bb40 con 0x7fae6c1628e0 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae50004d40 0x7fae500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6aa00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6b400640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae500039b0 0x7fae50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 -- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 msgr2=0x7fae50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 -- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae500039b0 msgr2=0x7fae50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae500039b0 0x7fae50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6be00640 1 -- 172.21.15.43:0/3925665497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae50003520 con 0x7fae50004d40 2024-04-26T20:26:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.693+0000 7fae6aa00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c1628e0 0x7fae50004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.694+0000 7fae6be00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae50004d40 0x7fae500051c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e170 tx=0x7fae50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32e166440cb6012 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64034020 con 0x7fae50004d40 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64030430 con 0x7fae50004d40 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae6a000640 1 -- 172.21.15.43:0/3925665497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64009c10 con 0x7fae50004d40 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae70a00640 1 -- 172.21.15.43:0/3925665497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae50004d40 msgr2=0x7fae500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae70a00640 1 --2- 172.21.15.43:0/3925665497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae50004d40 0x7fae500051c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e170 tx=0x7fae50005830 comp rx=0 tx=0).stop 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae6be00640 1 -- 172.21.15.43:0/3925665497 reap_dead start 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae70a00640 1 -- 172.21.15.43:0/3925665497 shutdown_connections 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.695+0000 7fae70a00640 1 -- 172.21.15.43:0/3925665497 >> 172.21.15.43:0/3925665497 conn(0x7fae6c068090 msgr2=0x7fae6c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.696+0000 7fae70a00640 1 -- 172.21.15.43:0/3925665497 shutdown_connections 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.696+0000 7fae70a00640 1 -- 172.21.15.43:0/3925665497 wait complete. 2024-04-26T20:26:15.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.697+0000 7fae70a00640 1 Processor -- start 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.698+0000 7fae70a00640 1 -- start start 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.698+0000 7fae70a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c0b3910 0x7fae6c1f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.698+0000 7fae70a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae70a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae6c1f19e0 0x7fae6c1f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae70a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0af200 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae70a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0aef20 con 0x7fae6c1f19e0 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae70a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0b00f0 con 0x7fae6c0b3910 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37778/0 (socket says 172.21.15.43:37778) 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 -- 172.21.15.43:0/861941941 learned_addr learned my addr 172.21.15.43:0/861941941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6be00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae6c1f19e0 0x7fae6c1f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 -- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c0b3910 msgr2=0x7fae6c1f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c0b3910 0x7fae6c1f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 -- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae6c1f19e0 msgr2=0x7fae6c1f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae6c1f19e0 0x7fae6c1f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.699+0000 7fae6aa00640 1 -- 172.21.15.43:0/861941941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c0cf280 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae6aa00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae540108e0 tx=0x7fae54010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400f040 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5400ec20 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5401c400 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c0cf520 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.700+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c0cfae0 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.701+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c069350 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.702+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae5401c5a0 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.703+0000 7fae63400640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.703+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fae5408c6e0 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.703+0000 7fae6b400640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:15.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.704+0000 7fae6b400640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae5c001eb0 tx=0x7fae5c000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:15.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.708+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae54018030 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.921+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae6c1622d0 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.922+0000 7fae63400640 1 -- 172.21.15.43:0/861941941 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fae5405a730 con 0x7fae6c0b3cf0 2024-04-26T20:26:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:26:15.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.924+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 msgr2=0x7fae4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.924+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae5c001eb0 tx=0x7fae5c000f50 comp rx=0 tx=0).stop 2024-04-26T20:26:15.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 msgr2=0x7fae6c0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:15.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fae540108e0 tx=0x7fae54010da0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 shutdown_connections 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae6c1f19e0 0x7fae6c1f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0b3cf0 0x7fae6c0ced40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 --2- 172.21.15.43:0/861941941 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fae6c0b3910 0x7fae6c1f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 >> 172.21.15.43:0/861941941 conn(0x7fae6c068090 msgr2=0x7fae6c161990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 shutdown_connections 2024-04-26T20:26:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:15.925+0000 7fae70a00640 1 -- 172.21.15.43:0/861941941 wait complete. 2024-04-26T20:26:15.945 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-26T20:26:15.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:15.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:26:16.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[38830]: fuse finished with error 0 and tester_r 0 2024-04-26T20:26:16.031 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:26:22.132 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:26:22.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:22.132 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T20:26:22.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.420+0000 7f934fe00640 1 Processor -- start 2024-04-26T20:26:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.421+0000 7f934fe00640 1 -- start start 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93500b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93500d1570 0x7f93500d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934fe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935015f360 con 0x7f93500b6ae0 2024-04-26T20:26:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f93500b0ad0 con 0x7f93500d1570 2024-04-26T20:26:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93500b0db0 con 0x7f93500afe90 2024-04-26T20:26:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934e000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f93500b6ae0 0x7f935015faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52462/0 (socket says 172.21.15.43:52462) 2024-04-26T20:26:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.422+0000 7f934e000640 1 -- 172.21.15.43:0/1093084503 learned_addr learned my addr 172.21.15.43:0/1093084503 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934ea00640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93500b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934f400640 1 --2- 172.21.15.43:0/1093084503 >> v2:172.21.15.164:3300/0 conn(0x7f93500d1570 0x7f93500d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409166266 0 0) 0x7f935015f360 con 0x7f93500b6ae0 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9330003520 con 0x7f93500b6ae0 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1206026377 0 0) 0x7f9330003520 con 0x7f93500b6ae0 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 msgr2=0x7f93500b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93500b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 >> v2:172.21.15.164:3300/0 conn(0x7f93500d1570 msgr2=0x7f93500d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 --2- 172.21.15.43:0/1093084503 >> v2:172.21.15.164:3300/0 conn(0x7f93500d1570 0x7f93500d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93500b1090 con 0x7f93500b6ae0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f93380019f0 con 0x7f93500b6ae0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 >> v1:172.21.15.43:6789/0 conn(0x7f93500b6ae0 legacy=0x7f935015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93300039b0 0x7f9330003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9330004800 0x7f9330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93500b0ad0 con 0x7f9330004800 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93500b0db0 con 0x7f93300039b0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.423+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f933000b9c0 con 0x7f93500afe90 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9330004800 0x7f9330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934ea00640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93300039b0 0x7f9330003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934f400640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 -- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 msgr2=0x7f93300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93500afe90 0x7f93300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 -- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93300039b0 msgr2=0x7f9330003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93300039b0 0x7f9330003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934e000640 1 -- 172.21.15.43:0/1093084503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9330003520 con 0x7f9330004800 2024-04-26T20:26:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.424+0000 7f934ea00640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93300039b0 0x7f9330003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.426+0000 7f934e000640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9330004800 0x7f9330004c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933800a8f0 tx=0x7f9330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3917afd91fd5df3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.426+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933800a4c0 con 0x7f9330004800 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.426+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933802f070 con 0x7f9330004800 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.426+0000 7f934d600640 1 -- 172.21.15.43:0/1093084503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933802bce0 con 0x7f9330004800 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 -- 172.21.15.43:0/1093084503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9330004800 msgr2=0x7f9330004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 --2- 172.21.15.43:0/1093084503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9330004800 0x7f9330004c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f933800a8f0 tx=0x7f9330005830 comp rx=0 tx=0).stop 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934f400640 1 -- 172.21.15.43:0/1093084503 reap_dead start 2024-04-26T20:26:22.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 -- 172.21.15.43:0/1093084503 shutdown_connections 2024-04-26T20:26:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 -- 172.21.15.43:0/1093084503 >> 172.21.15.43:0/1093084503 conn(0x7f9350075df0 msgr2=0x7f9350068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 -- 172.21.15.43:0/1093084503 shutdown_connections 2024-04-26T20:26:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.427+0000 7f934fe00640 1 -- 172.21.15.43:0/1093084503 wait complete. 2024-04-26T20:26:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.429+0000 7f934fe00640 1 Processor -- start 2024-04-26T20:26:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.429+0000 7f934fe00640 1 -- start start 2024-04-26T20:26:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f935006c5c0 0x7f93500c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935006c9a0 0x7f93501523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93500b0760 con 0x7f93500c8f60 2024-04-26T20:26:22.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9350070790 con 0x7f935006c9a0 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93500b0270 con 0x7f935006c5c0 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32958/0 (socket says 172.21.15.43:32958) 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934e000640 1 -- 172.21.15.43:0/3311593382 learned_addr learned my addr 172.21.15.43:0/3311593382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.430+0000 7f934ea00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f935006c5c0 0x7f93500c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934f400640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935006c9a0 0x7f93501523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 -- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f935006c5c0 msgr2=0x7f93500c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f935006c5c0 0x7f93500c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 -- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935006c9a0 msgr2=0x7f93501523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935006c9a0 0x7f93501523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 -- 172.21.15.43:0/3311593382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93500c94e0 con 0x7f93500c8f60 2024-04-26T20:26:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f934e000640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f933802b4b0 tx=0x7f9338001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933802bc90 con 0x7f93500c8f60 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.431+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933802be30 con 0x7f93500c8f60 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.432+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338034670 con 0x7f93500c8f60 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.432+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93500c97c0 con 0x7f93500c8f60 2024-04-26T20:26:22.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.432+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93500c9d80 con 0x7f93500c8f60 2024-04-26T20:26:22.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.433+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93500c9fc0 con 0x7f93500c8f60 2024-04-26T20:26:22.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.434+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9338034810 con 0x7f93500c8f60 2024-04-26T20:26:22.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.435+0000 7f9346a00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 0x7f932c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.435+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f93380ac380 con 0x7f93500c8f60 2024-04-26T20:26:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.435+0000 7f934ea00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 0x7f932c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.436+0000 7f934ea00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 0x7f932c06ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93400098b0 tx=0x7f9340008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.440+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93500c9fc0 con 0x7f93500c8f60 2024-04-26T20:26:22.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.645+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9350069a60 con 0x7f93500c8f60 2024-04-26T20:26:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.647+0000 7f9346a00640 1 -- 172.21.15.43:0/3311593382 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f933807a3e0 con 0x7f93500c8f60 2024-04-26T20:26:22.649 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:22.649 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.649+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 msgr2=0x7f932c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.649+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 0x7f932c06ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93400098b0 tx=0x7f9340008040 comp rx=0 tx=0).stop 2024-04-26T20:26:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 msgr2=0x7f9350151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f933802b4b0 tx=0x7f9338001dc0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 shutdown_connections 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f932c06c920 0x7f932c06ede0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f935006c9a0 0x7f93501523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93500c8f60 0x7f9350151eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 --2- 172.21.15.43:0/3311593382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f935006c5c0 0x7f93500c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 >> 172.21.15.43:0/3311593382 conn(0x7f9350075df0 msgr2=0x7f93500af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 shutdown_connections 2024-04-26T20:26:22.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.650+0000 7f934fe00640 1 -- 172.21.15.43:0/3311593382 wait complete. 2024-04-26T20:26:22.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.876+0000 7fdb05400640 1 Processor -- start 2024-04-26T20:26:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.876+0000 7fdb05400640 1 -- start start 2024-04-26T20:26:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.877+0000 7fdb05400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb000cd180 0x7fdb00072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.877+0000 7fdb05400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdb000ce480 0x7fdb0006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.877+0000 7fdb05400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb000cd560 con 0x7fdb0015dc50 2024-04-26T20:26:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.877+0000 7fdb05400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdb00072d00 con 0x7fdb000cd180 2024-04-26T20:26:22.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.877+0000 7fdb05400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdb00072fe0 con 0x7fdb000ce480 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaff400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdb0015dc50 0x7fdb000cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52478/0 (socket says 172.21.15.43:52478) 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaff400640 1 -- 172.21.15.43:0/464132708 learned_addr learned my addr 172.21.15.43:0/464132708 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaffe00640 1 --2- 172.21.15.43:0/464132708 >> v2:172.21.15.43:3300/0 conn(0x7fdb000cd180 0x7fdb00072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdb04a00640 1 --2- 172.21.15.43:0/464132708 >> v2:172.21.15.164:3300/0 conn(0x7fdb000ce480 0x7fdb0006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdafea00640 1 -- 172.21.15.43:0/464132708 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287071522 0 0) 0x7fdb000cd560 con 0x7fdb0015dc50 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdafea00640 1 -- 172.21.15.43:0/464132708 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdae4003520 con 0x7fdb0015dc50 2024-04-26T20:26:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaffe00640 1 -- 172.21.15.43:0/464132708 >> v2:172.21.15.164:3300/0 conn(0x7fdb000ce480 msgr2=0x7fdb0006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaffe00640 1 --2- 172.21.15.43:0/464132708 >> v2:172.21.15.164:3300/0 conn(0x7fdb000ce480 0x7fdb0006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.878+0000 7fdaffe00640 1 -- 172.21.15.43:0/464132708 >> v1:172.21.15.43:6789/0 conn(0x7fdb0015dc50 legacy=0x7fdb000cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.879+0000 7fdaffe00640 1 -- 172.21.15.43:0/464132708 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000732c0 con 0x7fdb000cd180 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdaffe00640 1 --2- 172.21.15.43:0/464132708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00072620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb000b39f0 tx=0x7fdaf002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce8a34ee6d414ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdafea00640 1 -- 172.21.15.43:0/464132708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0007740 con 0x7fdb000cd180 2024-04-26T20:26:22.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdafea00640 1 -- 172.21.15.43:0/464132708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf00078e0 con 0x7fdb000cd180 2024-04-26T20:26:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdafea00640 1 -- 172.21.15.43:0/464132708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0007cb0 con 0x7fdb000cd180 2024-04-26T20:26:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdb05400640 1 -- 172.21.15.43:0/464132708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 msgr2=0x7fdb00072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.880+0000 7fdb05400640 1 --2- 172.21.15.43:0/464132708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00072620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb000b39f0 tx=0x7fdaf002a680 comp rx=0 tx=0).stop 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 -- 172.21.15.43:0/464132708 shutdown_connections 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 --2- 172.21.15.43:0/464132708 >> v2:172.21.15.164:3300/0 conn(0x7fdb000ce480 0x7fdb0006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 --2- 172.21.15.43:0/464132708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00072620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 -- 172.21.15.43:0/464132708 >> 172.21.15.43:0/464132708 conn(0x7fdb00068090 msgr2=0x7fdb00068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 -- 172.21.15.43:0/464132708 shutdown_connections 2024-04-26T20:26:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.881+0000 7fdb05400640 1 -- 172.21.15.43:0/464132708 wait complete. 2024-04-26T20:26:22.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.883+0000 7fdb05400640 1 Processor -- start 2024-04-26T20:26:22.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.883+0000 7fdb05400640 1 -- start start 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb000ce480 0x7fdb00160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb0015dc50 0x7fdb00160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0015e0d0 con 0x7fdb000cd180 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00073da0 con 0x7fdb000ce480 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdb05400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0006e000 con 0x7fdb0015dc50 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32978/0 (socket says 172.21.15.43:32978) 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 -- 172.21.15.43:0/938848908 learned_addr learned my addr 172.21.15.43:0/938848908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 -- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb0015dc50 msgr2=0x7fdb00160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb0015dc50 0x7fdb00160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 -- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb000ce480 msgr2=0x7fdb00160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb000ce480 0x7fdb00160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.884+0000 7fdaffe00640 1 -- 172.21.15.43:0/938848908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000aa700 con 0x7fdb000cd180 2024-04-26T20:26:22.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdaffe00640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009e80 tx=0x7fdaf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0040070 con 0x7fdb000cd180 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf0031c40 con 0x7fdb000cd180 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb000aa9e0 con 0x7fdb000cd180 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0009450 con 0x7fdb000cd180 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.885+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb000aafa0 con 0x7fdb000cd180 2024-04-26T20:26:22.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.886+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb00069350 con 0x7fdb000cd180 2024-04-26T20:26:22.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.887+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaf0039040 con 0x7fdb000cd180 2024-04-26T20:26:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.888+0000 7fdaf7e00640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 0x7fdae006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.889+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdaf00ad410 con 0x7fdb000cd180 2024-04-26T20:26:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.889+0000 7fdaff400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 0x7fdae006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.889+0000 7fdaff400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 0x7fdae006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f60 tx=0x7fdaec0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:22.893+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdaf007af20 con 0x7fdb000cd180 2024-04-26T20:26:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.097+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb000d0670 con 0x7fdb000cd180 2024-04-26T20:26:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.101+0000 7fdaf7e00640 1 -- 172.21.15.43:0/938848908 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7fdaf007f220 con 0x7fdb000cd180 2024-04-26T20:26:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:23.103 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.103+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 msgr2=0x7fdae006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.103+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 0x7fdae006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f60 tx=0x7fdaec0078f0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.103+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 msgr2=0x7fdb00134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.103+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009e80 tx=0x7fdaf0004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 shutdown_connections 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdae006c940 0x7fdae006ee00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb0015dc50 0x7fdb00160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb000ce480 0x7fdb00160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 --2- 172.21.15.43:0/938848908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb000cd180 0x7fdb00134fc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 >> 172.21.15.43:0/938848908 conn(0x7fdb00068090 msgr2=0x7fdb00068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 shutdown_connections 2024-04-26T20:26:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.104+0000 7fdb05400640 1 -- 172.21.15.43:0/938848908 wait complete. 2024-04-26T20:26:23.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:26:23.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.330+0000 7fda2ce00640 1 Processor -- start 2024-04-26T20:26:23.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.330+0000 7fda2ce00640 1 -- start start 2024-04-26T20:26:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.331+0000 7fda2ce00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda280b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.331+0000 7fda2ce00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda280cf820 0x7fda280cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.331+0000 7fda2ce00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2806eaf0 con 0x7fda280d01c0 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda2ce00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fda2806f420 con 0x7fda280cf820 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda2ce00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fda2806edd0 con 0x7fda280cf440 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda26800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda280d01c0 0x7fda280b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52494/0 (socket says 172.21.15.43:52494) 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda26800640 1 -- 172.21.15.43:0/2639222764 learned_addr learned my addr 172.21.15.43:0/2639222764 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda1fe00640 1 --2- 172.21.15.43:0/2639222764 >> v2:172.21.15.43:3300/0 conn(0x7fda280cf820 0x7fda280cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda25e00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda280b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda25400640 1 -- 172.21.15.43:0/2639222764 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795169892 0 0) 0x7fda2806eaf0 con 0x7fda280d01c0 2024-04-26T20:26:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.332+0000 7fda25400640 1 -- 172.21.15.43:0/2639222764 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda0c003520 con 0x7fda280d01c0 2024-04-26T20:26:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.333+0000 7fda1fe00640 1 -- 172.21.15.43:0/2639222764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 msgr2=0x7fda280b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.333+0000 7fda1fe00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda280b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.333+0000 7fda1fe00640 1 -- 172.21.15.43:0/2639222764 >> v1:172.21.15.43:6789/0 conn(0x7fda280d01c0 legacy=0x7fda280b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.333+0000 7fda1fe00640 1 -- 172.21.15.43:0/2639222764 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2806f0b0 con 0x7fda280cf820 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.333+0000 7fda25e00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda280b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.334+0000 7fda1fe00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280cf820 0x7fda280cfc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda100097a0 tx=0x7fda1002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e274078f556c16c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.334+0000 7fda25400640 1 -- 172.21.15.43:0/2639222764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10037070 con 0x7fda280cf820 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.334+0000 7fda25400640 1 -- 172.21.15.43:0/2639222764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1002ae50 con 0x7fda280cf820 2024-04-26T20:26:23.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.334+0000 7fda25400640 1 -- 172.21.15.43:0/2639222764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10032660 con 0x7fda280cf820 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 -- 172.21.15.43:0/2639222764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280cf820 msgr2=0x7fda280cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280cf820 0x7fda280cfc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda100097a0 tx=0x7fda1002a100 comp rx=0 tx=0).stop 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 -- 172.21.15.43:0/2639222764 shutdown_connections 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280cf820 0x7fda280cfc80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 --2- 172.21.15.43:0/2639222764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda280b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 -- 172.21.15.43:0/2639222764 >> 172.21.15.43:0/2639222764 conn(0x7fda280b2900 msgr2=0x7fda280b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 -- 172.21.15.43:0/2639222764 shutdown_connections 2024-04-26T20:26:23.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.335+0000 7fda2ce00640 1 -- 172.21.15.43:0/2639222764 wait complete. 2024-04-26T20:26:23.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.337+0000 7fda2ce00640 1 Processor -- start 2024-04-26T20:26:23.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.337+0000 7fda2ce00640 1 -- start start 2024-04-26T20:26:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda2815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda280cf820 0x7fda28157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda2815e600 con 0x7fda280d01c0 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fda2806f8b0 con 0x7fda280cf820 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda2ce00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fda2815e780 con 0x7fda280cf440 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda26800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda26800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33000/0 (socket says 172.21.15.43:33000) 2024-04-26T20:26:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda26800640 1 -- 172.21.15.43:0/1351736942 learned_addr learned my addr 172.21.15.43:0/1351736942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:23.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.338+0000 7fda1fe00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda280cf820 0x7fda28157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda25e00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda2815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 -- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 msgr2=0x7fda2815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda2815c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 -- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda280cf820 msgr2=0x7fda28157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda280cf820 0x7fda28157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 -- 172.21.15.43:0/1351736942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda28158480 con 0x7fda280d01c0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda26800640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda2000cb40 tx=0x7fda2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20017070 con 0x7fda280d01c0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2000ac40 con 0x7fda280d01c0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2815bbc0 con 0x7fda280d01c0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.339+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20012400 con 0x7fda280d01c0 2024-04-26T20:26:23.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.340+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2815be80 con 0x7fda280d01c0 2024-04-26T20:26:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.341+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda28069350 con 0x7fda280d01c0 2024-04-26T20:26:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.342+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda2001a070 con 0x7fda280d01c0 2024-04-26T20:26:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.343+0000 7fda1e000640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 0x7fda0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.343+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda2008bc00 con 0x7fda280d01c0 2024-04-26T20:26:23.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.343+0000 7fda25e00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 0x7fda0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:23.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.344+0000 7fda25e00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 0x7fda0406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda281f3c40 tx=0x7fda18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.347+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda20059ce0 con 0x7fda280d01c0 2024-04-26T20:26:23.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:23.598+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fda28141370 con 0x7fda280d01c0 2024-04-26T20:26:24.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.154+0000 7fda1e000640 1 -- 172.21.15.43:0/1351736942 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fda200595e0 con 0x7fda280d01c0 2024-04-26T20:26:24.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.158+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 msgr2=0x7fda0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.158+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 0x7fda0406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda281f3c40 tx=0x7fda18008040 comp rx=0 tx=0).stop 2024-04-26T20:26:24.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.158+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 msgr2=0x7fda28144550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.158+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda2000cb40 tx=0x7fda2000a060 comp rx=0 tx=0).stop 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 shutdown_connections 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda0406c8f0 0x7fda0406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda280d01c0 0x7fda28144550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda280cf820 0x7fda28157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 --2- 172.21.15.43:0/1351736942 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda280cf440 0x7fda2815c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 >> 172.21.15.43:0/1351736942 conn(0x7fda280b2900 msgr2=0x7fda2815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 shutdown_connections 2024-04-26T20:26:24.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.159+0000 7fda2ce00640 1 -- 172.21.15.43:0/1351736942 wait complete. 2024-04-26T20:26:24.185 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T20:26:24.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:26:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.399+0000 7f9c7ae00640 1 Processor -- start 2024-04-26T20:26:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.400+0000 7f9c7ae00640 1 -- start start 2024-04-26T20:26:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.401+0000 7f9c7ae00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.401+0000 7f9c7ae00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9c74072b40 0x7f9c740b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.401+0000 7f9c7ae00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7415f920 con 0x7f9c7415f3f0 2024-04-26T20:26:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.401+0000 7f9c7ae00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7415fc00 con 0x7f9c74072b40 2024-04-26T20:26:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.401+0000 7f9c7ae00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7415fee0 con 0x7f9c74155cd0 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.402+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.402+0000 7f9c79000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9c74072b40 0x7f9c740b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.402+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:60146/0 (socket says 172.21.15.43:60146) 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.402+0000 7f9c79a00640 1 -- 172.21.15.43:0/1007582967 learned_addr learned my addr 172.21.15.43:0/1007582967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.403+0000 7f9c79000640 1 -- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 msgr2=0x7f9c74072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.403+0000 7f9c79000640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.403+0000 7f9c79000640 1 -- 172.21.15.43:0/1007582967 >> v1:172.21.15.164:6789/0 conn(0x7f9c7415f3f0 legacy=0x7f9c74162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.403+0000 7f9c79000640 1 -- 172.21.15.43:0/1007582967 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c741601c0 con 0x7f9c74072b40 2024-04-26T20:26:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.403+0000 7f9c79a00640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.404+0000 7f9c79000640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c74072b40 0x7f9c740b5400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c600097a0 tx=0x7f9c6002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7895eea114752179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c73e00640 1 -- 172.21.15.43:0/1007582967 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60037070 con 0x7f9c74072b40 2024-04-26T20:26:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c73e00640 1 -- 172.21.15.43:0/1007582967 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c60032440 con 0x7f9c74072b40 2024-04-26T20:26:24.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c73e00640 1 -- 172.21.15.43:0/1007582967 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60032810 con 0x7f9c74072b40 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 -- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c74072b40 msgr2=0x7f9c740b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c74072b40 0x7f9c740b5400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c600097a0 tx=0x7f9c6002a210 comp rx=0 tx=0).stop 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 -- 172.21.15.43:0/1007582967 shutdown_connections 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c74072b40 0x7f9c740b5400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/1007582967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c74072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.405+0000 7f9c7ae00640 1 -- 172.21.15.43:0/1007582967 >> 172.21.15.43:0/1007582967 conn(0x7f9c74068090 msgr2=0x7f9c740684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.406+0000 7f9c7ae00640 1 -- 172.21.15.43:0/1007582967 shutdown_connections 2024-04-26T20:26:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.406+0000 7f9c7ae00640 1 -- 172.21.15.43:0/1007582967 wait complete. 2024-04-26T20:26:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.407+0000 7f9c7ae00640 1 Processor -- start 2024-04-26T20:26:24.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.408+0000 7f9c7ae00640 1 -- start start 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c740b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 0x7f9c7415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74156230 con 0x7f9c74072b40 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741560b0 con 0x7f9c7415f3f0 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7ae00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c74163610 con 0x7f9c74155cd0 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33012/0 (socket says 172.21.15.43:33012) 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 -- 172.21.15.43:0/3474230482 learned_addr learned my addr 172.21.15.43:0/3474230482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79000640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c740b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c7a400640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 0x7f9c7415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 -- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 msgr2=0x7f9c740b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c740b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.409+0000 7f9c79a00640 1 -- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 msgr2=0x7f9c7415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c79a00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 0x7f9c7415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c79a00640 1 -- 172.21.15.43:0/3474230482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c74144070 con 0x7f9c74072b40 2024-04-26T20:26:24.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c7a400640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 0x7f9c7415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c79a00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400ea00 tx=0x7f9c6400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64017070 con 0x7f9c74072b40 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6400ca60 con 0x7f9c74072b40 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400ce10 con 0x7f9c74072b40 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c74144350 con 0x7f9c74072b40 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.410+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c74144910 con 0x7f9c74072b40 2024-04-26T20:26:24.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.412+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c74069d80 con 0x7f9c74072b40 2024-04-26T20:26:24.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.412+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c6401a070 con 0x7f9c74072b40 2024-04-26T20:26:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.413+0000 7f9c71600640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.414+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9c6408c1a0 con 0x7f9c74072b40 2024-04-26T20:26:24.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.414+0000 7f9c79000640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:24.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.415+0000 7f9c79000640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 0x7f9c5406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c60034000 tx=0x7f9c6002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.418+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c74069d80 con 0x7f9c74072b40 2024-04-26T20:26:24.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:24.663+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9c740506c0 con 0x7f9c74072b40 2024-04-26T20:26:25.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.169+0000 7f9c71600640 1 -- 172.21.15.43:0/3474230482 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f9c6405a280 con 0x7f9c74072b40 2024-04-26T20:26:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.172+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 msgr2=0x7f9c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.172+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 0x7f9c5406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c60034000 tx=0x7f9c6002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:25.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.172+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 msgr2=0x7f9c741563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.172+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400ea00 tx=0x7f9c6400eec0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 shutdown_connections 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9c5406c8f0 0x7f9c5406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c74155cd0 0x7f9c740b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7415f3f0 0x7f9c7415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 --2- 172.21.15.43:0/3474230482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c74072b40 0x7f9c741563b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 >> 172.21.15.43:0/3474230482 conn(0x7f9c74068090 msgr2=0x7f9c74162f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 shutdown_connections 2024-04-26T20:26:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.173+0000 7f9c7ae00640 1 -- 172.21.15.43:0/3474230482 wait complete. 2024-04-26T20:26:25.196 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:26:25.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:25.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.401+0000 7fdb5e400640 1 Processor -- start 2024-04-26T20:26:25.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.402+0000 7fdb5e400640 1 -- start start 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.403+0000 7fdb5e400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb580d1620 0x7fdb580d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.403+0000 7fdb5e400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb580d0480 con 0x7fdb580d1a00 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.403+0000 7fdb5e400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb580d0720 con 0x7fdb580d3810 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.403+0000 7fdb5e400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdb580d0ce0 con 0x7fdb580d1620 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb5d000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb580d1620 0x7fdb580d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb57e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdb580d1a00 0x7fdb580b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52504/0 (socket says 172.21.15.43:52504) 2024-04-26T20:26:25.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb57e00640 1 -- 172.21.15.43:0/689024705 learned_addr learned my addr 172.21.15.43:0/689024705 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:25.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb5d000640 1 -- 172.21.15.43:0/689024705 >> v1:172.21.15.164:6789/0 conn(0x7fdb580d3810 legacy=0x7fdb580b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:25.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb5d000640 1 -- 172.21.15.43:0/689024705 >> v1:172.21.15.43:6789/0 conn(0x7fdb580d1a00 legacy=0x7fdb580b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb5d000640 1 -- 172.21.15.43:0/689024705 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb580d0a00 con 0x7fdb580d1620 2024-04-26T20:26:25.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.404+0000 7fdb57400640 1 -- 172.21.15.43:0/689024705 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3856932919 0 0) 0x7fdb580d0480 con 0x7fdb580d1a00 2024-04-26T20:26:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.405+0000 7fdb5d000640 1 --2- 172.21.15.43:0/689024705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580d5750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3633cb94ac26d2f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb57400640 1 -- 172.21.15.43:0/689024705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c037070 con 0x7fdb580d1620 2024-04-26T20:26:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb57400640 1 -- 172.21.15.43:0/689024705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c02ae50 con 0x7fdb580d1620 2024-04-26T20:26:25.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb57400640 1 -- 172.21.15.43:0/689024705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c032660 con 0x7fdb580d1620 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 -- 172.21.15.43:0/689024705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 msgr2=0x7fdb580d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 --2- 172.21.15.43:0/689024705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580d5750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c02a140 comp rx=0 tx=0).stop 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 -- 172.21.15.43:0/689024705 shutdown_connections 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 --2- 172.21.15.43:0/689024705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580d5750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 -- 172.21.15.43:0/689024705 >> 172.21.15.43:0/689024705 conn(0x7fdb580d4c30 msgr2=0x7fdb580d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.406+0000 7fdb5e400640 1 -- 172.21.15.43:0/689024705 shutdown_connections 2024-04-26T20:26:25.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.407+0000 7fdb5e400640 1 -- 172.21.15.43:0/689024705 wait complete. 2024-04-26T20:26:25.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.408+0000 7fdb5e400640 1 Processor -- start 2024-04-26T20:26:25.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.408+0000 7fdb5e400640 1 -- start start 2024-04-26T20:26:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb580d1a00 0x7fdb581f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 0x7fdb581f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb580b5990 con 0x7fdb580d1620 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb580d1e60 con 0x7fdb580d3810 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.409+0000 7fdb5e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdb580b5b10 con 0x7fdb580d1a00 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33024/0 (socket says 172.21.15.43:33024) 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 -- 172.21.15.43:0/2995699825 learned_addr learned my addr 172.21.15.43:0/2995699825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb57e00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb580d1a00 0x7fdb581f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 -- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb580d1a00 msgr2=0x7fdb581f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb580d1a00 0x7fdb581f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 -- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 msgr2=0x7fdb581f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5da00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 0x7fdb581f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 0x7fdb581f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 -- 172.21.15.43:0/2995699825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5815e430 con 0x7fdb580d1620 2024-04-26T20:26:25.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5da00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 0x7fdb581f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.410+0000 7fdb5d000640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.411+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c042070 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.411+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c03e670 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.411+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c037040 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.411+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb580b8570 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.411+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb580b8dd0 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.412+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb58069350 con 0x7fdb580d1620 2024-04-26T20:26:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.413+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb4c045070 con 0x7fdb580d1620 2024-04-26T20:26:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.414+0000 7fdb54c00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 0x7fdb3c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.415+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdb4c0ac650 con 0x7fdb580d1620 2024-04-26T20:26:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.415+0000 7fdb57e00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 0x7fdb3c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.416+0000 7fdb57e00640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 0x7fdb3c06ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb581f7d20 tx=0x7fdb48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.419+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb4c07a940 con 0x7fdb580d1620 2024-04-26T20:26:25.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.669+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb580b94d0 con 0x7fdb580d1620 2024-04-26T20:26:25.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.670+0000 7fdb54c00640 1 -- 172.21.15.43:0/2995699825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fdb4c07a240 con 0x7fdb580d1620 2024-04-26T20:26:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:25.672 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:26:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.672+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 msgr2=0x7fdb3c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.672+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 0x7fdb3c06ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb581f7d20 tx=0x7fdb48009040 comp rx=0 tx=0).stop 2024-04-26T20:26:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.672+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 msgr2=0x7fdb580ba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.672+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 shutdown_connections 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdb3c06c940 0x7fdb3c06ee00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb580d1a00 0x7fdb581f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb580d3810 0x7fdb581f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 --2- 172.21.15.43:0/2995699825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb580d1620 0x7fdb580ba310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 >> 172.21.15.43:0/2995699825 conn(0x7fdb580d4c30 msgr2=0x7fdb5806df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 shutdown_connections 2024-04-26T20:26:25.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.673+0000 7fdb5e400640 1 -- 172.21.15.43:0/2995699825 wait complete. 2024-04-26T20:26:25.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.895+0000 7faeb0e00640 1 Processor -- start 2024-04-26T20:26:25.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.895+0000 7faeb0e00640 1 -- start start 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.897+0000 7faeb0e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faeac0b5b80 0x7faeac0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.897+0000 7faeb0e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac0d0480 con 0x7faeac0d2800 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.897+0000 7faeb0e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac0d0720 con 0x7faeac0d1620 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.897+0000 7faeb0e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7faeac0d0ce0 con 0x7faeac0b5b80 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeabe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faeac0d2800 0x7faeac0d2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41356/0 (socket says 172.21.15.43:41356) 2024-04-26T20:26:25.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeabe00640 1 -- 172.21.15.43:0/456347774 learned_addr learned my addr 172.21.15.43:0/456347774 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaaa00640 1 --2- 172.21.15.43:0/456347774 >> v2:172.21.15.43:3300/0 conn(0x7faeac0b5b80 0x7faeac0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaa000640 1 -- 172.21.15.43:0/456347774 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47820148 0 0) 0x7faeac0d0480 con 0x7faeac0d2800 2024-04-26T20:26:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaa000640 1 -- 172.21.15.43:0/456347774 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae90003520 con 0x7faeac0d2800 2024-04-26T20:26:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaaa00640 1 -- 172.21.15.43:0/456347774 >> v1:172.21.15.164:6789/0 conn(0x7faeac0d1620 legacy=0x7faeac0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaaa00640 1 -- 172.21.15.43:0/456347774 >> v1:172.21.15.43:6789/0 conn(0x7faeac0d2800 legacy=0x7faeac0d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.898+0000 7faeaaa00640 1 -- 172.21.15.43:0/456347774 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac0d0a00 con 0x7faeac0b5b80 2024-04-26T20:26:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.899+0000 7faeaaa00640 1 --2- 172.21.15.43:0/456347774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b5b80 0x7faeac0d5750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae980097a0 tx=0x7fae9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95ea8318570be24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.900+0000 7faeaa000640 1 -- 172.21.15.43:0/456347774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98037070 con 0x7faeac0b5b80 2024-04-26T20:26:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.900+0000 7faeaa000640 1 -- 172.21.15.43:0/456347774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae98032440 con 0x7faeac0b5b80 2024-04-26T20:26:25.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.900+0000 7faeaa000640 1 -- 172.21.15.43:0/456347774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98032810 con 0x7faeac0b5b80 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.900+0000 7faeb0e00640 1 -- 172.21.15.43:0/456347774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b5b80 msgr2=0x7faeac0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.900+0000 7faeb0e00640 1 --2- 172.21.15.43:0/456347774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b5b80 0x7faeac0d5750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae980097a0 tx=0x7fae9802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.901+0000 7faeb0e00640 1 -- 172.21.15.43:0/456347774 shutdown_connections 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.901+0000 7faeb0e00640 1 --2- 172.21.15.43:0/456347774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b5b80 0x7faeac0d5750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.901+0000 7faeb0e00640 1 -- 172.21.15.43:0/456347774 >> 172.21.15.43:0/456347774 conn(0x7faeac0d4c30 msgr2=0x7faeac0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:25.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.901+0000 7faeb0e00640 1 -- 172.21.15.43:0/456347774 shutdown_connections 2024-04-26T20:26:25.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.901+0000 7faeb0e00640 1 -- 172.21.15.43:0/456347774 wait complete. 2024-04-26T20:26:25.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.902+0000 7faeb0e00640 1 Processor -- start 2024-04-26T20:26:25.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.903+0000 7faeb0e00640 1 -- start start 2024-04-26T20:26:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 0x7faeac14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 0x7faeac0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0d3520 con 0x7faeac0b9e50 2024-04-26T20:26:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0d33a0 con 0x7faeac0d1620 2024-04-26T20:26:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeb0e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faeac0d36a0 con 0x7faeac0d2800 2024-04-26T20:26:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54490/0 (socket says 172.21.15.43:54490) 2024-04-26T20:26:25.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 -- 172.21.15.43:0/2208106250 learned_addr learned my addr 172.21.15.43:0/2208106250 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:25.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeab400640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 0x7faeac14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeaaa00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 0x7faeac0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 -- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 msgr2=0x7faeac0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 0x7faeac0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 -- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 msgr2=0x7faeac14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 0x7faeac14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeabe00640 1 -- 172.21.15.43:0/2208106250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac0b2590 con 0x7faeac0b9e50 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeab400640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 0x7faeac14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.904+0000 7faeaaa00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 0x7faeac0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faeabe00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faea400caa0 tx=0x7faea400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4017070 con 0x7faeac0b9e50 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea400ac00 con 0x7faeac0b9e50 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4012400 con 0x7faeac0b9e50 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac0b9370 con 0x7faeac0b9e50 2024-04-26T20:26:25.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.905+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac159560 con 0x7faeac0b9e50 2024-04-26T20:26:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.907+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac06d3f0 con 0x7faeac0b9e50 2024-04-26T20:26:25.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.907+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faea401a070 con 0x7faeac0b9e50 2024-04-26T20:26:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.908+0000 7faea3400640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 0x7fae8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.909+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faea408bbd0 con 0x7faeac0b9e50 2024-04-26T20:26:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.909+0000 7faeab400640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 0x7fae8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:25.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.909+0000 7faeab400640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 0x7fae8c06ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae9c000f60 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:25.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:25.913+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faeac06d3f0 con 0x7faeac0b9e50 2024-04-26T20:26:26.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.120+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeac069350 con 0x7faeac0b9e50 2024-04-26T20:26:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.123+0000 7faea3400640 1 -- 172.21.15.43:0/2208106250 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7faea4059cc0 con 0x7faeac0b9e50 2024-04-26T20:26:26.125 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:26.125 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 msgr2=0x7fae8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 0x7fae8c06ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae9c000f60 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 msgr2=0x7faeac0ba300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faea400caa0 tx=0x7faea400cf60 comp rx=0 tx=0).stop 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 shutdown_connections 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fae8c06c920 0x7fae8c06ede0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faeac0d2800 0x7faeac0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faeac0b9e50 0x7faeac0ba300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 --2- 172.21.15.43:0/2208106250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faeac0d1620 0x7faeac14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.125+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 >> 172.21.15.43:0/2208106250 conn(0x7faeac0d4c30 msgr2=0x7faeac0d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.126+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 shutdown_connections 2024-04-26T20:26:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.126+0000 7faeb0e00640 1 -- 172.21.15.43:0/2208106250 wait complete. 2024-04-26T20:26:26.146 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:26:26.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:26.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.360+0000 7f278e800640 1 Processor -- start 2024-04-26T20:26:26.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.361+0000 7f278e800640 1 -- start start 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f278815cab0 0x7f27880afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f27880b0020 0x7f27880d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f27880d1080 0x7f27880ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f27880d0900 con 0x7f27880b0020 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f27881635a0 con 0x7f27880d1080 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.362+0000 7f278e800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f27880b0650 con 0x7f278815cab0 2024-04-26T20:26:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f27880b0020 0x7f27880d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f27880b0020 0x7f27880d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54500/0 (socket says 172.21.15.43:54500) 2024-04-26T20:26:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 -- 172.21.15.43:0/1976816652 learned_addr learned my addr 172.21.15.43:0/1976816652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2787e00640 1 --2- 172.21.15.43:0/1976816652 >> v2:172.21.15.164:3300/0 conn(0x7f27880d1080 0x7f27880ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2787400640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f278815cab0 0x7f27880afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 -- 172.21.15.43:0/1976816652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f278815cab0 msgr2=0x7f27880afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f278815cab0 0x7f27880afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 -- 172.21.15.43:0/1976816652 >> v2:172.21.15.164:3300/0 conn(0x7f27880d1080 msgr2=0x7f27880ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 --2- 172.21.15.43:0/1976816652 >> v2:172.21.15.164:3300/0 conn(0x7f27880d1080 0x7f27880ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.363+0000 7f2786a00640 1 -- 172.21.15.43:0/1976816652 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2788163a30 con 0x7f27880b0020 2024-04-26T20:26:26.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f2786a00640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27880b0020 0x7f27880d0b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2788163510 tx=0x7f277402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259be359a46a007 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f2786000640 1 -- 172.21.15.43:0/1976816652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774037070 con 0x7f27880b0020 2024-04-26T20:26:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f2786000640 1 -- 172.21.15.43:0/1976816652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2774032440 con 0x7f27880b0020 2024-04-26T20:26:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f2786000640 1 -- 172.21.15.43:0/1976816652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774032810 con 0x7f27880b0020 2024-04-26T20:26:26.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f278e800640 1 -- 172.21.15.43:0/1976816652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27880b0020 msgr2=0x7f27880d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.365+0000 7f278e800640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27880b0020 0x7f27880d0b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2788163510 tx=0x7f277402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 -- 172.21.15.43:0/1976816652 shutdown_connections 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 --2- 172.21.15.43:0/1976816652 >> v2:172.21.15.164:3300/0 conn(0x7f27880d1080 0x7f27880ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27880b0020 0x7f27880d0b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 --2- 172.21.15.43:0/1976816652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f278815cab0 0x7f27880afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 -- 172.21.15.43:0/1976816652 >> 172.21.15.43:0/1976816652 conn(0x7f27880b2900 msgr2=0x7f27880b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 -- 172.21.15.43:0/1976816652 shutdown_connections 2024-04-26T20:26:26.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.366+0000 7f278e800640 1 -- 172.21.15.43:0/1976816652 wait complete. 2024-04-26T20:26:26.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.368+0000 7f278e800640 1 Processor -- start 2024-04-26T20:26:26.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.368+0000 7f278e800640 1 -- start start 2024-04-26T20:26:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27880b0020 0x7f278814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27880d1080 0x7f2788072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2788163230 con 0x7f278815cab0 2024-04-26T20:26:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2788162f50 con 0x7f27880d1080 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f278e800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f278815ce90 con 0x7f27880b0020 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f2787e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f2787e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54508/0 (socket says 172.21.15.43:54508) 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f2787e00640 1 -- 172.21.15.43:0/4173222874 learned_addr learned my addr 172.21.15.43:0/4173222874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f2787400640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27880b0020 0x7f278814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.369+0000 7f2786a00640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27880d1080 0x7f2788072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 -- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27880b0020 msgr2=0x7f278814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27880b0020 0x7f278814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 -- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27880d1080 msgr2=0x7f2788072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27880d1080 0x7f2788072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 -- 172.21.15.43:0/4173222874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278814daf0 con 0x7f278815cab0 2024-04-26T20:26:26.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f2787e00640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f278000d800 tx=0x7f278000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780018020 con 0x7f278815cab0 2024-04-26T20:26:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27800096c0 con 0x7f278815cab0 2024-04-26T20:26:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780009a90 con 0x7f278815cab0 2024-04-26T20:26:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.370+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2788158b50 con 0x7f278815cab0 2024-04-26T20:26:26.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.371+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2788159110 con 0x7f278815cab0 2024-04-26T20:26:26.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.372+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2788069350 con 0x7f278815cab0 2024-04-26T20:26:26.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.373+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f278001b020 con 0x7f278815cab0 2024-04-26T20:26:26.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.374+0000 7f277f400640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 0x7f276806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.374+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f278008c120 con 0x7f278815cab0 2024-04-26T20:26:26.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.374+0000 7f2787400640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 0x7f276806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.375+0000 7f2787400640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 0x7f276806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f27780099c0 tx=0x7f2778003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.378+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2780022030 con 0x7f278815cab0 2024-04-26T20:26:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.583+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27880b1f20 con 0x7f276806c8f0 2024-04-26T20:26:26.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.594+0000 7f277f400640 1 -- 172.21.15.43:0/4173222874 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332574 (secure 0 0 0) 0x7f27880b1f20 con 0x7f276806c8f0 2024-04-26T20:26:26.600 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:26.602 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 msgr2=0x7f276806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 0x7f276806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f27780099c0 tx=0x7f2778003990 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 msgr2=0x7f278815f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f278000d800 tx=0x7f278000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 shutdown_connections 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f276806c8f0 0x7f276806edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f278815cab0 0x7f278815f570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27880d1080 0x7f2788072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 --2- 172.21.15.43:0/4173222874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27880b0020 0x7f278814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.600+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 >> 172.21.15.43:0/4173222874 conn(0x7f27880b2900 msgr2=0x7f278815e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.601+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 shutdown_connections 2024-04-26T20:26:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.601+0000 7f278e800640 1 -- 172.21.15.43:0/4173222874 wait complete. 2024-04-26T20:26:26.622 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-26T20:26:24.866785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227728,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746992,"statfs":{"total":773094113280,"available":772860919808,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004830"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347756+0000","last_change":"2024-04-26T20:25:56.984031+0000","last_active":"2024-04-26T20:26:03.347756+0000","last_peered":"2024-04-26T20:26:03.347756+0000","last_clean":"2024-04-26T20:26:03.347756+0000","last_became_active":"2024-04-26T20:25:56.983640+0000","last_became_peered":"2024-04-26T20:25:56.983640+0000","last_unstale":"2024-04-26T20:26:03.347756+0000","last_undegraded":"2024-04-26T20:26:03.347756+0000","last_fullsized":"2024-04-26T20:26:03.347756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:04.709694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094688+0000","last_change":"2024-04-26T20:25:57.999359+0000","last_active":"2024-04-26T20:26:03.094688+0000","last_peered":"2024-04-26T20:26:03.094688+0000","last_clean":"2024-04-26T20:26:03.094688+0000","last_became_active":"2024-04-26T20:25:57.999124+0000","last_became_peered":"2024-04-26T20:25:57.999124+0000","last_unstale":"2024-04-26T20:26:03.094688+0000","last_undegraded":"2024-04-26T20:26:03.094688+0000","last_fullsized":"2024-04-26T20:26:03.094688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092925+0000","last_change":"2024-04-26T20:26:02.417831+0000","last_active":"2024-04-26T20:26:03.092925+0000","last_peered":"2024-04-26T20:26:03.092925+0000","last_clean":"2024-04-26T20:26:03.092925+0000","last_became_active":"2024-04-26T20:26:02.417256+0000","last_became_peered":"2024-04-26T20:26:02.417256+0000","last_unstale":"2024-04-26T20:26:03.092925+0000","last_undegraded":"2024-04-26T20:26:03.092925+0000","last_fullsized":"2024-04-26T20:26:03.092925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094948+0000","last_change":"2024-04-26T20:25:56.981647+0000","last_active":"2024-04-26T20:26:03.094948+0000","last_peered":"2024-04-26T20:26:03.094948+0000","last_clean":"2024-04-26T20:26:03.094948+0000","last_became_active":"2024-04-26T20:25:56.980997+0000","last_became_peered":"2024-04-26T20:25:56.980997+0000","last_unstale":"2024-04-26T20:26:03.094948+0000","last_undegraded":"2024-04-26T20:26:03.094948+0000","last_fullsized":"2024-04-26T20:26:03.094948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347273+0000","last_change":"2024-04-26T20:25:58.006366+0000","last_active":"2024-04-26T20:26:03.347273+0000","last_peered":"2024-04-26T20:26:03.347273+0000","last_clean":"2024-04-26T20:26:03.347273+0000","last_became_active":"2024-04-26T20:25:58.005106+0000","last_became_peered":"2024-04-26T20:25:58.005106+0000","last_unstale":"2024-04-26T20:26:03.347273+0000","last_undegraded":"2024-04-26T20:26:03.347273+0000","last_fullsized":"2024-04-26T20:26:03.347273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050650+0000","last_change":"2024-04-26T20:26:02.077388+0000","last_active":"2024-04-26T20:26:03.050650+0000","last_peered":"2024-04-26T20:26:03.050650+0000","last_clean":"2024-04-26T20:26:03.050650+0000","last_became_active":"2024-04-26T20:26:02.076618+0000","last_became_peered":"2024-04-26T20:26:02.076618+0000","last_unstale":"2024-04-26T20:26:03.050650+0000","last_undegraded":"2024-04-26T20:26:03.050650+0000","last_fullsized":"2024-04-26T20:26:03.050650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:59.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348208+0000","last_change":"2024-04-26T20:25:56.993261+0000","last_active":"2024-04-26T20:26:03.348208+0000","last_peered":"2024-04-26T20:26:03.348208+0000","last_clean":"2024-04-26T20:26:03.348208+0000","last_became_active":"2024-04-26T20:25:56.992657+0000","last_became_peered":"2024-04-26T20:25:56.992657+0000","last_unstale":"2024-04-26T20:26:03.348208+0000","last_undegraded":"2024-04-26T20:26:03.348208+0000","last_fullsized":"2024-04-26T20:26:03.348208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346205+0000","last_change":"2024-04-26T20:25:58.008848+0000","last_active":"2024-04-26T20:26:03.346205+0000","last_peered":"2024-04-26T20:26:03.346205+0000","last_clean":"2024-04-26T20:26:03.346205+0000","last_became_active":"2024-04-26T20:25:58.008316+0000","last_became_peered":"2024-04-26T20:25:58.008316+0000","last_unstale":"2024-04-26T20:26:03.346205+0000","last_undegraded":"2024-04-26T20:26:03.346205+0000","last_fullsized":"2024-04-26T20:26:03.346205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051568+0000","last_change":"2024-04-26T20:26:02.080666+0000","last_active":"2024-04-26T20:26:03.051568+0000","last_peered":"2024-04-26T20:26:03.051568+0000","last_clean":"2024-04-26T20:26:03.051568+0000","last_became_active":"2024-04-26T20:26:02.080286+0000","last_became_peered":"2024-04-26T20:26:02.080286+0000","last_unstale":"2024-04-26T20:26:03.051568+0000","last_undegraded":"2024-04-26T20:26:03.051568+0000","last_fullsized":"2024-04-26T20:26:03.051568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347297+0000","last_change":"2024-04-26T20:25:56.993036+0000","last_active":"2024-04-26T20:26:03.347297+0000","last_peered":"2024-04-26T20:26:03.347297+0000","last_clean":"2024-04-26T20:26:03.347297+0000","last_became_active":"2024-04-26T20:25:56.992712+0000","last_became_peered":"2024-04-26T20:25:56.992712+0000","last_unstale":"2024-04-26T20:26:03.347297+0000","last_undegraded":"2024-04-26T20:26:03.347297+0000","last_fullsized":"2024-04-26T20:26:03.347297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:11.781113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347210+0000","last_change":"2024-04-26T20:25:58.005509+0000","last_active":"2024-04-26T20:26:03.347210+0000","last_peered":"2024-04-26T20:26:03.347210+0000","last_clean":"2024-04-26T20:26:03.347210+0000","last_became_active":"2024-04-26T20:25:58.005266+0000","last_became_peered":"2024-04-26T20:25:58.005266+0000","last_unstale":"2024-04-26T20:26:03.347210+0000","last_undegraded":"2024-04-26T20:26:03.347210+0000","last_fullsized":"2024-04-26T20:26:03.347210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050541+0000","last_change":"2024-04-26T20:26:02.076146+0000","last_active":"2024-04-26T20:26:03.050541+0000","last_peered":"2024-04-26T20:26:03.050541+0000","last_clean":"2024-04-26T20:26:03.050541+0000","last_became_active":"2024-04-26T20:26:02.075597+0000","last_became_peered":"2024-04-26T20:26:02.075597+0000","last_unstale":"2024-04-26T20:26:03.050541+0000","last_undegraded":"2024-04-26T20:26:03.050541+0000","last_fullsized":"2024-04-26T20:26:03.050541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:49.620083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349354+0000","last_change":"2024-04-26T20:25:57.995411+0000","last_active":"2024-04-26T20:26:03.349354+0000","last_peered":"2024-04-26T20:26:03.349354+0000","last_clean":"2024-04-26T20:26:03.349354+0000","last_became_active":"2024-04-26T20:25:57.993865+0000","last_became_peered":"2024-04-26T20:25:57.993865+0000","last_unstale":"2024-04-26T20:26:03.349354+0000","last_undegraded":"2024-04-26T20:26:03.349354+0000","last_fullsized":"2024-04-26T20:26:03.349354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347802+0000","last_change":"2024-04-26T20:25:56.986121+0000","last_active":"2024-04-26T20:26:03.347802+0000","last_peered":"2024-04-26T20:26:03.347802+0000","last_clean":"2024-04-26T20:26:03.347802+0000","last_became_active":"2024-04-26T20:25:56.985836+0000","last_became_peered":"2024-04-26T20:25:56.985836+0000","last_unstale":"2024-04-26T20:26:03.347802+0000","last_undegraded":"2024-04-26T20:26:03.347802+0000","last_fullsized":"2024-04-26T20:26:03.347802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:01.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049987+0000","last_change":"2024-04-26T20:26:02.079776+0000","last_active":"2024-04-26T20:26:03.049987+0000","last_peered":"2024-04-26T20:26:03.049987+0000","last_clean":"2024-04-26T20:26:03.049987+0000","last_became_active":"2024-04-26T20:26:02.079355+0000","last_became_peered":"2024-04-26T20:26:02.079355+0000","last_unstale":"2024-04-26T20:26:03.049987+0000","last_undegraded":"2024-04-26T20:26:03.049987+0000","last_fullsized":"2024-04-26T20:26:03.049987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:32.933389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093856+0000","last_change":"2024-04-26T20:25:57.990924+0000","last_active":"2024-04-26T20:26:03.093856+0000","last_peered":"2024-04-26T20:26:03.093856+0000","last_clean":"2024-04-26T20:26:03.093856+0000","last_became_active":"2024-04-26T20:25:57.990615+0000","last_became_peered":"2024-04-26T20:25:57.990615+0000","last_unstale":"2024-04-26T20:26:03.093856+0000","last_undegraded":"2024-04-26T20:26:03.093856+0000","last_fullsized":"2024-04-26T20:26:03.093856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345060+0000","last_change":"2024-04-26T20:25:56.985786+0000","last_active":"2024-04-26T20:26:03.345060+0000","last_peered":"2024-04-26T20:26:03.345060+0000","last_clean":"2024-04-26T20:26:03.345060+0000","last_became_active":"2024-04-26T20:25:56.981498+0000","last_became_peered":"2024-04-26T20:25:56.981498+0000","last_unstale":"2024-04-26T20:26:03.345060+0000","last_undegraded":"2024-04-26T20:26:03.345060+0000","last_fullsized":"2024-04-26T20:26:03.345060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049606+0000","last_change":"2024-04-26T20:26:02.075573+0000","last_active":"2024-04-26T20:26:03.049606+0000","last_peered":"2024-04-26T20:26:03.049606+0000","last_clean":"2024-04-26T20:26:03.049606+0000","last_became_active":"2024-04-26T20:26:02.075012+0000","last_became_peered":"2024-04-26T20:26:02.075012+0000","last_unstale":"2024-04-26T20:26:03.049606+0000","last_undegraded":"2024-04-26T20:26:03.049606+0000","last_fullsized":"2024-04-26T20:26:03.049606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.401666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093255+0000","last_change":"2024-04-26T20:25:58.011840+0000","last_active":"2024-04-26T20:26:03.093255+0000","last_peered":"2024-04-26T20:26:03.093255+0000","last_clean":"2024-04-26T20:26:03.093255+0000","last_became_active":"2024-04-26T20:25:58.011346+0000","last_became_peered":"2024-04-26T20:25:58.011346+0000","last_unstale":"2024-04-26T20:26:03.093255+0000","last_undegraded":"2024-04-26T20:26:03.093255+0000","last_fullsized":"2024-04-26T20:26:03.093255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344179+0000","last_change":"2024-04-26T20:25:56.986011+0000","last_active":"2024-04-26T20:26:03.344179+0000","last_peered":"2024-04-26T20:26:03.344179+0000","last_clean":"2024-04-26T20:26:03.344179+0000","last_became_active":"2024-04-26T20:25:56.985728+0000","last_became_peered":"2024-04-26T20:25:56.985728+0000","last_unstale":"2024-04-26T20:26:03.344179+0000","last_undegraded":"2024-04-26T20:26:03.344179+0000","last_fullsized":"2024-04-26T20:26:03.344179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347438+0000","last_change":"2024-04-26T20:26:02.401041+0000","last_active":"2024-04-26T20:26:03.347438+0000","last_peered":"2024-04-26T20:26:03.347438+0000","last_clean":"2024-04-26T20:26:03.347438+0000","last_became_active":"2024-04-26T20:26:02.400108+0000","last_became_peered":"2024-04-26T20:26:02.400108+0000","last_unstale":"2024-04-26T20:26:03.347438+0000","last_undegraded":"2024-04-26T20:26:03.347438+0000","last_fullsized":"2024-04-26T20:26:03.347438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049644+0000","last_change":"2024-04-26T20:25:57.985787+0000","last_active":"2024-04-26T20:26:03.049644+0000","last_peered":"2024-04-26T20:26:03.049644+0000","last_clean":"2024-04-26T20:26:03.049644+0000","last_became_active":"2024-04-26T20:25:57.985432+0000","last_became_peered":"2024-04-26T20:25:57.985432+0000","last_unstale":"2024-04-26T20:26:03.049644+0000","last_undegraded":"2024-04-26T20:26:03.049644+0000","last_fullsized":"2024-04-26T20:26:03.049644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:37.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345394+0000","last_change":"2024-04-26T20:25:56.983944+0000","last_active":"2024-04-26T20:26:03.345394+0000","last_peered":"2024-04-26T20:26:03.345394+0000","last_clean":"2024-04-26T20:26:03.345394+0000","last_became_active":"2024-04-26T20:25:56.983642+0000","last_became_peered":"2024-04-26T20:25:56.983642+0000","last_unstale":"2024-04-26T20:26:03.345394+0000","last_undegraded":"2024-04-26T20:26:03.345394+0000","last_fullsized":"2024-04-26T20:26:03.345394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:12.976926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050080+0000","last_change":"2024-04-26T20:26:02.082133+0000","last_active":"2024-04-26T20:26:03.050080+0000","last_peered":"2024-04-26T20:26:03.050080+0000","last_clean":"2024-04-26T20:26:03.050080+0000","last_became_active":"2024-04-26T20:26:02.081547+0000","last_became_peered":"2024-04-26T20:26:02.081547+0000","last_unstale":"2024-04-26T20:26:03.050080+0000","last_undegraded":"2024-04-26T20:26:03.050080+0000","last_fullsized":"2024-04-26T20:26:03.050080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:39.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347003+0000","last_change":"2024-04-26T20:25:57.989909+0000","last_active":"2024-04-26T20:26:03.347003+0000","last_peered":"2024-04-26T20:26:03.347003+0000","last_clean":"2024-04-26T20:26:03.347003+0000","last_became_active":"2024-04-26T20:25:57.989626+0000","last_became_peered":"2024-04-26T20:25:57.989626+0000","last_unstale":"2024-04-26T20:26:03.347003+0000","last_undegraded":"2024-04-26T20:26:03.347003+0000","last_fullsized":"2024-04-26T20:26:03.347003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351956+0000","last_change":"2024-04-26T20:25:56.985557+0000","last_active":"2024-04-26T20:26:03.351956+0000","last_peered":"2024-04-26T20:26:03.351956+0000","last_clean":"2024-04-26T20:26:03.351956+0000","last_became_active":"2024-04-26T20:25:56.985217+0000","last_became_peered":"2024-04-26T20:25:56.985217+0000","last_unstale":"2024-04-26T20:26:03.351956+0000","last_undegraded":"2024-04-26T20:26:03.351956+0000","last_fullsized":"2024-04-26T20:26:03.351956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050981+0000","last_change":"2024-04-26T20:26:02.405217+0000","last_active":"2024-04-26T20:26:03.050981+0000","last_peered":"2024-04-26T20:26:03.050981+0000","last_clean":"2024-04-26T20:26:03.050981+0000","last_became_active":"2024-04-26T20:26:02.404124+0000","last_became_peered":"2024-04-26T20:26:02.404124+0000","last_unstale":"2024-04-26T20:26:03.050981+0000","last_undegraded":"2024-04-26T20:26:03.050981+0000","last_fullsized":"2024-04-26T20:26:03.050981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:29.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095109+0000","last_change":"2024-04-26T20:25:57.999075+0000","last_active":"2024-04-26T20:26:03.095109+0000","last_peered":"2024-04-26T20:26:03.095109+0000","last_clean":"2024-04-26T20:26:03.095109+0000","last_became_active":"2024-04-26T20:25:57.998666+0000","last_became_peered":"2024-04-26T20:25:57.998666+0000","last_unstale":"2024-04-26T20:26:03.095109+0000","last_undegraded":"2024-04-26T20:26:03.095109+0000","last_fullsized":"2024-04-26T20:26:03.095109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346736+0000","last_change":"2024-04-26T20:25:56.977150+0000","last_active":"2024-04-26T20:26:03.346736+0000","last_peered":"2024-04-26T20:26:03.346736+0000","last_clean":"2024-04-26T20:26:03.346736+0000","last_became_active":"2024-04-26T20:25:56.976895+0000","last_became_peered":"2024-04-26T20:25:56.976895+0000","last_unstale":"2024-04-26T20:26:03.346736+0000","last_undegraded":"2024-04-26T20:26:03.346736+0000","last_fullsized":"2024-04-26T20:26:03.346736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093148+0000","last_change":"2024-04-26T20:26:02.400496+0000","last_active":"2024-04-26T20:26:03.093148+0000","last_peered":"2024-04-26T20:26:03.093148+0000","last_clean":"2024-04-26T20:26:03.093148+0000","last_became_active":"2024-04-26T20:26:02.400002+0000","last_became_peered":"2024-04-26T20:26:02.400002+0000","last_unstale":"2024-04-26T20:26:03.093148+0000","last_undegraded":"2024-04-26T20:26:03.093148+0000","last_fullsized":"2024-04-26T20:26:03.093148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345843+0000","last_change":"2024-04-26T20:25:58.009127+0000","last_active":"2024-04-26T20:26:03.345843+0000","last_peered":"2024-04-26T20:26:03.345843+0000","last_clean":"2024-04-26T20:26:03.345843+0000","last_became_active":"2024-04-26T20:25:58.006542+0000","last_became_peered":"2024-04-26T20:25:58.006542+0000","last_unstale":"2024-04-26T20:26:03.345843+0000","last_undegraded":"2024-04-26T20:26:03.345843+0000","last_fullsized":"2024-04-26T20:26:03.345843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348511+0000","last_change":"2024-04-26T20:25:56.992958+0000","last_active":"2024-04-26T20:26:03.348511+0000","last_peered":"2024-04-26T20:26:03.348511+0000","last_clean":"2024-04-26T20:26:03.348511+0000","last_became_active":"2024-04-26T20:25:56.992326+0000","last_became_peered":"2024-04-26T20:25:56.992326+0000","last_unstale":"2024-04-26T20:26:03.348511+0000","last_undegraded":"2024-04-26T20:26:03.348511+0000","last_fullsized":"2024-04-26T20:26:03.348511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346294+0000","last_change":"2024-04-26T20:26:02.406897+0000","last_active":"2024-04-26T20:26:03.346294+0000","last_peered":"2024-04-26T20:26:03.346294+0000","last_clean":"2024-04-26T20:26:03.346294+0000","last_became_active":"2024-04-26T20:26:02.402553+0000","last_became_peered":"2024-04-26T20:26:02.402553+0000","last_unstale":"2024-04-26T20:26:03.346294+0000","last_undegraded":"2024-04-26T20:26:03.346294+0000","last_fullsized":"2024-04-26T20:26:03.346294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347458+0000","last_change":"2024-04-26T20:25:58.003889+0000","last_active":"2024-04-26T20:26:03.347458+0000","last_peered":"2024-04-26T20:26:03.347458+0000","last_clean":"2024-04-26T20:26:03.347458+0000","last_became_active":"2024-04-26T20:25:58.003578+0000","last_became_peered":"2024-04-26T20:25:58.003578+0000","last_unstale":"2024-04-26T20:26:03.347458+0000","last_undegraded":"2024-04-26T20:26:03.347458+0000","last_fullsized":"2024-04-26T20:26:03.347458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346054+0000","last_change":"2024-04-26T20:25:56.975939+0000","last_active":"2024-04-26T20:26:03.346054+0000","last_peered":"2024-04-26T20:26:03.346054+0000","last_clean":"2024-04-26T20:26:03.346054+0000","last_became_active":"2024-04-26T20:25:56.975669+0000","last_became_peered":"2024-04-26T20:25:56.975669+0000","last_unstale":"2024-04-26T20:26:03.346054+0000","last_undegraded":"2024-04-26T20:26:03.346054+0000","last_fullsized":"2024-04-26T20:26:03.346054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344705+0000","last_change":"2024-04-26T20:26:02.079073+0000","last_active":"2024-04-26T20:26:03.344705+0000","last_peered":"2024-04-26T20:26:03.344705+0000","last_clean":"2024-04-26T20:26:03.344705+0000","last_became_active":"2024-04-26T20:26:02.074779+0000","last_became_peered":"2024-04-26T20:26:02.074779+0000","last_unstale":"2024-04-26T20:26:03.344705+0000","last_undegraded":"2024-04-26T20:26:03.344705+0000","last_fullsized":"2024-04-26T20:26:03.344705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051836+0000","last_change":"2024-04-26T20:25:57.988308+0000","last_active":"2024-04-26T20:26:03.051836+0000","last_peered":"2024-04-26T20:26:03.051836+0000","last_clean":"2024-04-26T20:26:03.051836+0000","last_became_active":"2024-04-26T20:25:57.984254+0000","last_became_peered":"2024-04-26T20:25:57.984254+0000","last_unstale":"2024-04-26T20:26:03.051836+0000","last_undegraded":"2024-04-26T20:26:03.051836+0000","last_fullsized":"2024-04-26T20:26:03.051836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:26.721241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348724+0000","last_change":"2024-04-26T20:25:56.980841+0000","last_active":"2024-04-26T20:26:03.348724+0000","last_peered":"2024-04-26T20:26:03.348724+0000","last_clean":"2024-04-26T20:26:03.348724+0000","last_became_active":"2024-04-26T20:25:56.980215+0000","last_became_peered":"2024-04-26T20:25:56.980215+0000","last_unstale":"2024-04-26T20:26:03.348724+0000","last_undegraded":"2024-04-26T20:26:03.348724+0000","last_fullsized":"2024-04-26T20:26:03.348724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345720+0000","last_change":"2024-04-26T20:26:02.078478+0000","last_active":"2024-04-26T20:26:03.345720+0000","last_peered":"2024-04-26T20:26:03.345720+0000","last_clean":"2024-04-26T20:26:03.345720+0000","last_became_active":"2024-04-26T20:26:02.078121+0000","last_became_peered":"2024-04-26T20:26:02.078121+0000","last_unstale":"2024-04-26T20:26:03.345720+0000","last_undegraded":"2024-04-26T20:26:03.345720+0000","last_fullsized":"2024-04-26T20:26:03.345720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343920+0000","last_change":"2024-04-26T20:25:58.001121+0000","last_active":"2024-04-26T20:26:03.343920+0000","last_peered":"2024-04-26T20:26:03.343920+0000","last_clean":"2024-04-26T20:26:03.343920+0000","last_became_active":"2024-04-26T20:25:58.000412+0000","last_became_peered":"2024-04-26T20:25:58.000412+0000","last_unstale":"2024-04-26T20:26:03.343920+0000","last_undegraded":"2024-04-26T20:26:03.343920+0000","last_fullsized":"2024-04-26T20:26:03.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048086+0000","last_change":"2024-04-26T20:25:56.972300+0000","last_active":"2024-04-26T20:26:03.048086+0000","last_peered":"2024-04-26T20:26:03.048086+0000","last_clean":"2024-04-26T20:26:03.048086+0000","last_became_active":"2024-04-26T20:25:56.972020+0000","last_became_peered":"2024-04-26T20:25:56.972020+0000","last_unstale":"2024-04-26T20:26:03.048086+0000","last_undegraded":"2024-04-26T20:26:03.048086+0000","last_fullsized":"2024-04-26T20:26:03.048086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347521+0000","last_change":"2024-04-26T20:26:02.077923+0000","last_active":"2024-04-26T20:26:03.347521+0000","last_peered":"2024-04-26T20:26:03.347521+0000","last_clean":"2024-04-26T20:26:03.347521+0000","last_became_active":"2024-04-26T20:26:02.077483+0000","last_became_peered":"2024-04-26T20:26:02.077483+0000","last_unstale":"2024-04-26T20:26:03.347521+0000","last_undegraded":"2024-04-26T20:26:03.347521+0000","last_fullsized":"2024-04-26T20:26:03.347521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344678+0000","last_change":"2024-04-26T20:25:57.984633+0000","last_active":"2024-04-26T20:26:03.344678+0000","last_peered":"2024-04-26T20:26:03.344678+0000","last_clean":"2024-04-26T20:26:03.344678+0000","last_became_active":"2024-04-26T20:25:57.984262+0000","last_became_peered":"2024-04-26T20:25:57.984262+0000","last_unstale":"2024-04-26T20:26:03.344678+0000","last_undegraded":"2024-04-26T20:26:03.344678+0000","last_fullsized":"2024-04-26T20:26:03.344678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:03.002879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345671+0000","last_change":"2024-04-26T20:25:56.989467+0000","last_active":"2024-04-26T20:26:03.345671+0000","last_peered":"2024-04-26T20:26:03.345671+0000","last_clean":"2024-04-26T20:26:03.345671+0000","last_became_active":"2024-04-26T20:25:56.989183+0000","last_became_peered":"2024-04-26T20:25:56.989183+0000","last_unstale":"2024-04-26T20:26:03.345671+0000","last_undegraded":"2024-04-26T20:26:03.345671+0000","last_fullsized":"2024-04-26T20:26:03.345671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:11.650840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346929+0000","last_change":"2024-04-26T20:26:02.414455+0000","last_active":"2024-04-26T20:26:03.346929+0000","last_peered":"2024-04-26T20:26:03.346929+0000","last_clean":"2024-04-26T20:26:03.346929+0000","last_became_active":"2024-04-26T20:26:02.414064+0000","last_became_peered":"2024-04-26T20:26:02.414064+0000","last_unstale":"2024-04-26T20:26:03.346929+0000","last_undegraded":"2024-04-26T20:26:03.346929+0000","last_fullsized":"2024-04-26T20:26:03.346929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093496+0000","last_change":"2024-04-26T20:25:58.008847+0000","last_active":"2024-04-26T20:26:03.093496+0000","last_peered":"2024-04-26T20:26:03.093496+0000","last_clean":"2024-04-26T20:26:03.093496+0000","last_became_active":"2024-04-26T20:25:58.008549+0000","last_became_peered":"2024-04-26T20:25:58.008549+0000","last_unstale":"2024-04-26T20:26:03.093496+0000","last_undegraded":"2024-04-26T20:26:03.093496+0000","last_fullsized":"2024-04-26T20:26:03.093496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345379+0000","last_change":"2024-04-26T20:25:56.981771+0000","last_active":"2024-04-26T20:26:03.345379+0000","last_peered":"2024-04-26T20:26:03.345379+0000","last_clean":"2024-04-26T20:26:03.345379+0000","last_became_active":"2024-04-26T20:25:56.981406+0000","last_became_peered":"2024-04-26T20:25:56.981406+0000","last_unstale":"2024-04-26T20:26:03.345379+0000","last_undegraded":"2024-04-26T20:26:03.345379+0000","last_fullsized":"2024-04-26T20:26:03.345379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345981+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:26:03.345981+0000","last_peered":"2024-04-26T20:26:03.345981+0000","last_clean":"2024-04-26T20:26:03.345981+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:26:03.345981+0000","last_undegraded":"2024-04-26T20:26:03.345981+0000","last_fullsized":"2024-04-26T20:26:03.345981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351315+0000","last_change":"2024-04-26T20:26:02.407562+0000","last_active":"2024-04-26T20:26:03.351315+0000","last_peered":"2024-04-26T20:26:03.351315+0000","last_clean":"2024-04-26T20:26:03.351315+0000","last_became_active":"2024-04-26T20:26:02.406288+0000","last_became_peered":"2024-04-26T20:26:02.406288+0000","last_unstale":"2024-04-26T20:26:03.351315+0000","last_undegraded":"2024-04-26T20:26:03.351315+0000","last_fullsized":"2024-04-26T20:26:03.351315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347732+0000","last_change":"2024-04-26T20:25:58.014416+0000","last_active":"2024-04-26T20:26:03.347732+0000","last_peered":"2024-04-26T20:26:03.347732+0000","last_clean":"2024-04-26T20:26:03.347732+0000","last_became_active":"2024-04-26T20:25:58.014128+0000","last_became_peered":"2024-04-26T20:25:58.014128+0000","last_unstale":"2024-04-26T20:26:03.347732+0000","last_undegraded":"2024-04-26T20:26:03.347732+0000","last_fullsized":"2024-04-26T20:26:03.347732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:22.089301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048942+0000","last_change":"2024-04-26T20:25:56.974272+0000","last_active":"2024-04-26T20:26:03.048942+0000","last_peered":"2024-04-26T20:26:03.048942+0000","last_clean":"2024-04-26T20:26:03.048942+0000","last_became_active":"2024-04-26T20:25:56.972999+0000","last_became_peered":"2024-04-26T20:25:56.972999+0000","last_unstale":"2024-04-26T20:26:03.048942+0000","last_undegraded":"2024-04-26T20:26:03.048942+0000","last_fullsized":"2024-04-26T20:26:03.048942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349430+0000","last_change":"2024-04-26T20:26:02.402839+0000","last_active":"2024-04-26T20:26:03.349430+0000","last_peered":"2024-04-26T20:26:03.349430+0000","last_clean":"2024-04-26T20:26:03.349430+0000","last_became_active":"2024-04-26T20:26:02.401630+0000","last_became_peered":"2024-04-26T20:26:02.401630+0000","last_unstale":"2024-04-26T20:26:03.349430+0000","last_undegraded":"2024-04-26T20:26:03.349430+0000","last_fullsized":"2024-04-26T20:26:03.349430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346830+0000","last_change":"2024-04-26T20:25:58.009012+0000","last_active":"2024-04-26T20:26:03.346830+0000","last_peered":"2024-04-26T20:26:03.346830+0000","last_clean":"2024-04-26T20:26:03.346830+0000","last_became_active":"2024-04-26T20:25:58.008594+0000","last_became_peered":"2024-04-26T20:25:58.008594+0000","last_unstale":"2024-04-26T20:26:03.346830+0000","last_undegraded":"2024-04-26T20:26:03.346830+0000","last_fullsized":"2024-04-26T20:26:03.346830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344764+0000","last_change":"2024-04-26T20:25:56.972043+0000","last_active":"2024-04-26T20:26:03.344764+0000","last_peered":"2024-04-26T20:26:03.344764+0000","last_clean":"2024-04-26T20:26:03.344764+0000","last_became_active":"2024-04-26T20:25:56.971671+0000","last_became_peered":"2024-04-26T20:25:56.971671+0000","last_unstale":"2024-04-26T20:26:03.344764+0000","last_undegraded":"2024-04-26T20:26:03.344764+0000","last_fullsized":"2024-04-26T20:26:03.344764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346513+0000","last_change":"2024-04-26T20:26:02.073728+0000","last_active":"2024-04-26T20:26:03.346513+0000","last_peered":"2024-04-26T20:26:03.346513+0000","last_clean":"2024-04-26T20:26:03.346513+0000","last_became_active":"2024-04-26T20:26:02.073238+0000","last_became_peered":"2024-04-26T20:26:02.073238+0000","last_unstale":"2024-04-26T20:26:03.346513+0000","last_undegraded":"2024-04-26T20:26:03.346513+0000","last_fullsized":"2024-04-26T20:26:03.346513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350204+0000","last_change":"2024-04-26T20:25:58.011539+0000","last_active":"2024-04-26T20:26:03.350204+0000","last_peered":"2024-04-26T20:26:03.350204+0000","last_clean":"2024-04-26T20:26:03.350204+0000","last_became_active":"2024-04-26T20:25:58.011256+0000","last_became_peered":"2024-04-26T20:25:58.011256+0000","last_unstale":"2024-04-26T20:26:03.350204+0000","last_undegraded":"2024-04-26T20:26:03.350204+0000","last_fullsized":"2024-04-26T20:26:03.350204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:02.292377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048755+0000","last_change":"2024-04-26T20:25:56.972332+0000","last_active":"2024-04-26T20:26:03.048755+0000","last_peered":"2024-04-26T20:26:03.048755+0000","last_clean":"2024-04-26T20:26:03.048755+0000","last_became_active":"2024-04-26T20:25:56.972075+0000","last_became_peered":"2024-04-26T20:25:56.972075+0000","last_unstale":"2024-04-26T20:26:03.048755+0000","last_undegraded":"2024-04-26T20:26:03.048755+0000","last_fullsized":"2024-04-26T20:26:03.048755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346282+0000","last_change":"2024-04-26T20:26:02.414970+0000","last_active":"2024-04-26T20:26:03.346282+0000","last_peered":"2024-04-26T20:26:03.346282+0000","last_clean":"2024-04-26T20:26:03.346282+0000","last_became_active":"2024-04-26T20:26:02.414093+0000","last_became_peered":"2024-04-26T20:26:02.414093+0000","last_unstale":"2024-04-26T20:26:03.346282+0000","last_undegraded":"2024-04-26T20:26:03.346282+0000","last_fullsized":"2024-04-26T20:26:03.346282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348952+0000","last_change":"2024-04-26T20:25:58.014604+0000","last_active":"2024-04-26T20:26:03.348952+0000","last_peered":"2024-04-26T20:26:03.348952+0000","last_clean":"2024-04-26T20:26:03.348952+0000","last_became_active":"2024-04-26T20:25:58.014239+0000","last_became_peered":"2024-04-26T20:25:58.014239+0000","last_unstale":"2024-04-26T20:26:03.348952+0000","last_undegraded":"2024-04-26T20:26:03.348952+0000","last_fullsized":"2024-04-26T20:26:03.348952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:24.555699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050237+0000","last_change":"2024-04-26T20:25:56.970496+0000","last_active":"2024-04-26T20:26:03.050237+0000","last_peered":"2024-04-26T20:26:03.050237+0000","last_clean":"2024-04-26T20:26:03.050237+0000","last_became_active":"2024-04-26T20:25:56.970174+0000","last_became_peered":"2024-04-26T20:25:56.970174+0000","last_unstale":"2024-04-26T20:26:03.050237+0000","last_undegraded":"2024-04-26T20:26:03.050237+0000","last_fullsized":"2024-04-26T20:26:03.050237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:25.599135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049091+0000","last_change":"2024-04-26T20:26:02.082060+0000","last_active":"2024-04-26T20:26:03.049091+0000","last_peered":"2024-04-26T20:26:03.049091+0000","last_clean":"2024-04-26T20:26:03.049091+0000","last_became_active":"2024-04-26T20:26:02.081533+0000","last_became_peered":"2024-04-26T20:26:02.081533+0000","last_unstale":"2024-04-26T20:26:03.049091+0000","last_undegraded":"2024-04-26T20:26:03.049091+0000","last_fullsized":"2024-04-26T20:26:03.049091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:02.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346324+0000","last_change":"2024-04-26T20:25:58.008801+0000","last_active":"2024-04-26T20:26:03.346324+0000","last_peered":"2024-04-26T20:26:03.346324+0000","last_clean":"2024-04-26T20:26:03.346324+0000","last_became_active":"2024-04-26T20:25:58.008479+0000","last_became_peered":"2024-04-26T20:25:58.008479+0000","last_unstale":"2024-04-26T20:26:03.346324+0000","last_undegraded":"2024-04-26T20:26:03.346324+0000","last_fullsized":"2024-04-26T20:26:03.346324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047755+0000","last_change":"2024-04-26T20:25:56.968431+0000","last_active":"2024-04-26T20:26:03.047755+0000","last_peered":"2024-04-26T20:26:03.047755+0000","last_clean":"2024-04-26T20:26:03.047755+0000","last_became_active":"2024-04-26T20:25:56.967495+0000","last_became_peered":"2024-04-26T20:25:56.967495+0000","last_unstale":"2024-04-26T20:26:03.047755+0000","last_undegraded":"2024-04-26T20:26:03.047755+0000","last_fullsized":"2024-04-26T20:26:03.047755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346734+0000","last_change":"2024-04-26T20:26:02.078801+0000","last_active":"2024-04-26T20:26:03.346734+0000","last_peered":"2024-04-26T20:26:03.346734+0000","last_clean":"2024-04-26T20:26:03.346734+0000","last_became_active":"2024-04-26T20:26:02.078343+0000","last_became_peered":"2024-04-26T20:26:02.078343+0000","last_unstale":"2024-04-26T20:26:03.346734+0000","last_undegraded":"2024-04-26T20:26:03.346734+0000","last_fullsized":"2024-04-26T20:26:03.346734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350880+0000","last_change":"2024-04-26T20:25:57.996180+0000","last_active":"2024-04-26T20:26:03.350880+0000","last_peered":"2024-04-26T20:26:03.350880+0000","last_clean":"2024-04-26T20:26:03.350880+0000","last_became_active":"2024-04-26T20:25:57.990792+0000","last_became_peered":"2024-04-26T20:25:57.990792+0000","last_unstale":"2024-04-26T20:26:03.350880+0000","last_undegraded":"2024-04-26T20:26:03.350880+0000","last_fullsized":"2024-04-26T20:26:03.350880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349887+0000","last_change":"2024-04-26T20:25:56.974115+0000","last_active":"2024-04-26T20:26:03.349887+0000","last_peered":"2024-04-26T20:26:03.349887+0000","last_clean":"2024-04-26T20:26:03.349887+0000","last_became_active":"2024-04-26T20:25:56.973801+0000","last_became_peered":"2024-04-26T20:25:56.973801+0000","last_unstale":"2024-04-26T20:26:03.349887+0000","last_undegraded":"2024-04-26T20:26:03.349887+0000","last_fullsized":"2024-04-26T20:26:03.349887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095178+0000","last_change":"2024-04-26T20:26:02.413590+0000","last_active":"2024-04-26T20:26:03.095178+0000","last_peered":"2024-04-26T20:26:03.095178+0000","last_clean":"2024-04-26T20:26:03.095178+0000","last_became_active":"2024-04-26T20:26:02.413104+0000","last_became_peered":"2024-04-26T20:26:02.413104+0000","last_unstale":"2024-04-26T20:26:03.095178+0000","last_undegraded":"2024-04-26T20:26:03.095178+0000","last_fullsized":"2024-04-26T20:26:03.095178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:35.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347726+0000","last_change":"2024-04-26T20:25:58.014095+0000","last_active":"2024-04-26T20:26:03.347726+0000","last_peered":"2024-04-26T20:26:03.347726+0000","last_clean":"2024-04-26T20:26:03.347726+0000","last_became_active":"2024-04-26T20:25:58.012813+0000","last_became_peered":"2024-04-26T20:25:58.012813+0000","last_unstale":"2024-04-26T20:26:03.347726+0000","last_undegraded":"2024-04-26T20:26:03.347726+0000","last_fullsized":"2024-04-26T20:26:03.347726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348384+0000","last_change":"2024-04-26T20:25:56.987103+0000","last_active":"2024-04-26T20:26:03.348384+0000","last_peered":"2024-04-26T20:26:03.348384+0000","last_clean":"2024-04-26T20:26:03.348384+0000","last_became_active":"2024-04-26T20:25:56.986766+0000","last_became_peered":"2024-04-26T20:25:56.986766+0000","last_unstale":"2024-04-26T20:26:03.348384+0000","last_undegraded":"2024-04-26T20:26:03.348384+0000","last_fullsized":"2024-04-26T20:26:03.348384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094102+0000","last_change":"2024-04-26T20:26:02.400999+0000","last_active":"2024-04-26T20:26:03.094102+0000","last_peered":"2024-04-26T20:26:03.094102+0000","last_clean":"2024-04-26T20:26:03.094102+0000","last_became_active":"2024-04-26T20:26:02.399994+0000","last_became_peered":"2024-04-26T20:26:02.399994+0000","last_unstale":"2024-04-26T20:26:03.094102+0000","last_undegraded":"2024-04-26T20:26:03.094102+0000","last_fullsized":"2024-04-26T20:26:03.094102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:05.311868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345529+0000","last_change":"2024-04-26T20:25:57.996909+0000","last_active":"2024-04-26T20:26:03.345529+0000","last_peered":"2024-04-26T20:26:03.345529+0000","last_clean":"2024-04-26T20:26:03.345529+0000","last_became_active":"2024-04-26T20:25:57.996635+0000","last_became_peered":"2024-04-26T20:25:57.996635+0000","last_unstale":"2024-04-26T20:26:03.345529+0000","last_undegraded":"2024-04-26T20:26:03.345529+0000","last_fullsized":"2024-04-26T20:26:03.345529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346908+0000","last_change":"2024-04-26T20:25:56.974019+0000","last_active":"2024-04-26T20:26:03.346908+0000","last_peered":"2024-04-26T20:26:03.346908+0000","last_clean":"2024-04-26T20:26:03.346908+0000","last_became_active":"2024-04-26T20:25:56.971170+0000","last_became_peered":"2024-04-26T20:25:56.971170+0000","last_unstale":"2024-04-26T20:26:03.346908+0000","last_undegraded":"2024-04-26T20:26:03.346908+0000","last_fullsized":"2024-04-26T20:26:03.346908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347253+0000","last_change":"2024-04-26T20:26:02.402012+0000","last_active":"2024-04-26T20:26:03.347253+0000","last_peered":"2024-04-26T20:26:03.347253+0000","last_clean":"2024-04-26T20:26:03.347253+0000","last_became_active":"2024-04-26T20:26:02.400605+0000","last_became_peered":"2024-04-26T20:26:02.400605+0000","last_unstale":"2024-04-26T20:26:03.347253+0000","last_undegraded":"2024-04-26T20:26:03.347253+0000","last_fullsized":"2024-04-26T20:26:03.347253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347168+0000","last_change":"2024-04-26T20:25:58.001764+0000","last_active":"2024-04-26T20:26:03.347168+0000","last_peered":"2024-04-26T20:26:03.347168+0000","last_clean":"2024-04-26T20:26:03.347168+0000","last_became_active":"2024-04-26T20:25:58.001502+0000","last_became_peered":"2024-04-26T20:25:58.001502+0000","last_unstale":"2024-04-26T20:26:03.347168+0000","last_undegraded":"2024-04-26T20:26:03.347168+0000","last_fullsized":"2024-04-26T20:26:03.347168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:16.030394+0000","last_change":"2024-04-26T20:25:56.984259+0000","last_active":"2024-04-26T20:26:16.030394+0000","last_peered":"2024-04-26T20:26:16.030394+0000","last_clean":"2024-04-26T20:26:16.030394+0000","last_became_active":"2024-04-26T20:25:56.983974+0000","last_became_peered":"2024-04-26T20:25:56.983974+0000","last_unstale":"2024-04-26T20:26:16.030394+0000","last_undegraded":"2024-04-26T20:26:16.030394+0000","last_fullsized":"2024-04-26T20:26:16.030394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345142+0000","last_change":"2024-04-26T20:26:02.080802+0000","last_active":"2024-04-26T20:26:03.345142+0000","last_peered":"2024-04-26T20:26:03.345142+0000","last_clean":"2024-04-26T20:26:03.345142+0000","last_became_active":"2024-04-26T20:26:02.079790+0000","last_became_peered":"2024-04-26T20:26:02.079790+0000","last_unstale":"2024-04-26T20:26:03.345142+0000","last_undegraded":"2024-04-26T20:26:03.345142+0000","last_fullsized":"2024-04-26T20:26:03.345142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050516+0000","last_change":"2024-04-26T20:25:57.999695+0000","last_active":"2024-04-26T20:26:03.050516+0000","last_peered":"2024-04-26T20:26:03.050516+0000","last_clean":"2024-04-26T20:26:03.050516+0000","last_became_active":"2024-04-26T20:25:57.999476+0000","last_became_peered":"2024-04-26T20:25:57.999476+0000","last_unstale":"2024-04-26T20:26:03.050516+0000","last_undegraded":"2024-04-26T20:26:03.050516+0000","last_fullsized":"2024-04-26T20:26:03.050516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344567+0000","last_change":"2024-04-26T20:25:56.984463+0000","last_active":"2024-04-26T20:26:03.344567+0000","last_peered":"2024-04-26T20:26:03.344567+0000","last_clean":"2024-04-26T20:26:03.344567+0000","last_became_active":"2024-04-26T20:25:56.984176+0000","last_became_peered":"2024-04-26T20:25:56.984176+0000","last_unstale":"2024-04-26T20:26:03.344567+0000","last_undegraded":"2024-04-26T20:26:03.344567+0000","last_fullsized":"2024-04-26T20:26:03.344567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051617+0000","last_change":"2024-04-26T20:26:02.079338+0000","last_active":"2024-04-26T20:26:03.051617+0000","last_peered":"2024-04-26T20:26:03.051617+0000","last_clean":"2024-04-26T20:26:03.051617+0000","last_became_active":"2024-04-26T20:26:02.078830+0000","last_became_peered":"2024-04-26T20:26:02.078830+0000","last_unstale":"2024-04-26T20:26:03.051617+0000","last_undegraded":"2024-04-26T20:26:03.051617+0000","last_fullsized":"2024-04-26T20:26:03.051617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347553+0000","last_change":"2024-04-26T20:25:58.008157+0000","last_active":"2024-04-26T20:26:03.347553+0000","last_peered":"2024-04-26T20:26:03.347553+0000","last_clean":"2024-04-26T20:26:03.347553+0000","last_became_active":"2024-04-26T20:25:58.005542+0000","last_became_peered":"2024-04-26T20:25:58.005542+0000","last_unstale":"2024-04-26T20:26:03.347553+0000","last_undegraded":"2024-04-26T20:26:03.347553+0000","last_fullsized":"2024-04-26T20:26:03.347553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348327+0000","last_change":"2024-04-26T20:25:56.987547+0000","last_active":"2024-04-26T20:26:03.348327+0000","last_peered":"2024-04-26T20:26:03.348327+0000","last_clean":"2024-04-26T20:26:03.348327+0000","last_became_active":"2024-04-26T20:25:56.985375+0000","last_became_peered":"2024-04-26T20:25:56.985375+0000","last_unstale":"2024-04-26T20:26:03.348327+0000","last_undegraded":"2024-04-26T20:26:03.348327+0000","last_fullsized":"2024-04-26T20:26:03.348327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093045+0000","last_change":"2024-04-26T20:26:02.073521+0000","last_active":"2024-04-26T20:26:03.093045+0000","last_peered":"2024-04-26T20:26:03.093045+0000","last_clean":"2024-04-26T20:26:03.093045+0000","last_became_active":"2024-04-26T20:26:02.073134+0000","last_became_peered":"2024-04-26T20:26:02.073134+0000","last_unstale":"2024-04-26T20:26:03.093045+0000","last_undegraded":"2024-04-26T20:26:03.093045+0000","last_fullsized":"2024-04-26T20:26:03.093045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:06.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348491+0000","last_change":"2024-04-26T20:25:58.011178+0000","last_active":"2024-04-26T20:26:03.348491+0000","last_peered":"2024-04-26T20:26:03.348491+0000","last_clean":"2024-04-26T20:26:03.348491+0000","last_became_active":"2024-04-26T20:25:58.010833+0000","last_became_peered":"2024-04-26T20:25:58.010833+0000","last_unstale":"2024-04-26T20:26:03.348491+0000","last_undegraded":"2024-04-26T20:26:03.348491+0000","last_fullsized":"2024-04-26T20:26:03.348491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351644+0000","last_change":"2024-04-26T20:25:56.986174+0000","last_active":"2024-04-26T20:26:03.351644+0000","last_peered":"2024-04-26T20:26:03.351644+0000","last_clean":"2024-04-26T20:26:03.351644+0000","last_became_active":"2024-04-26T20:25:56.985869+0000","last_became_peered":"2024-04-26T20:25:56.985869+0000","last_unstale":"2024-04-26T20:26:03.351644+0000","last_undegraded":"2024-04-26T20:26:03.351644+0000","last_fullsized":"2024-04-26T20:26:03.351644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093097+0000","last_change":"2024-04-26T20:26:02.413559+0000","last_active":"2024-04-26T20:26:03.093097+0000","last_peered":"2024-04-26T20:26:03.093097+0000","last_clean":"2024-04-26T20:26:03.093097+0000","last_became_active":"2024-04-26T20:26:02.413096+0000","last_became_peered":"2024-04-26T20:26:02.413096+0000","last_unstale":"2024-04-26T20:26:03.093097+0000","last_undegraded":"2024-04-26T20:26:03.093097+0000","last_fullsized":"2024-04-26T20:26:03.093097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:44.245008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346456+0000","last_change":"2024-04-26T20:26:02.413921+0000","last_active":"2024-04-26T20:26:03.346456+0000","last_peered":"2024-04-26T20:26:03.346456+0000","last_clean":"2024-04-26T20:26:03.346456+0000","last_became_active":"2024-04-26T20:26:02.413388+0000","last_became_peered":"2024-04-26T20:26:02.413388+0000","last_unstale":"2024-04-26T20:26:03.346456+0000","last_undegraded":"2024-04-26T20:26:03.346456+0000","last_fullsized":"2024-04-26T20:26:03.346456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345276+0000","last_change":"2024-04-26T20:25:58.004583+0000","last_active":"2024-04-26T20:26:03.345276+0000","last_peered":"2024-04-26T20:26:03.345276+0000","last_clean":"2024-04-26T20:26:03.345276+0000","last_became_active":"2024-04-26T20:25:58.004228+0000","last_became_peered":"2024-04-26T20:25:58.004228+0000","last_unstale":"2024-04-26T20:26:03.345276+0000","last_undegraded":"2024-04-26T20:26:03.345276+0000","last_fullsized":"2024-04-26T20:26:03.345276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346675+0000","last_change":"2024-04-26T20:25:56.985910+0000","last_active":"2024-04-26T20:26:03.346675+0000","last_peered":"2024-04-26T20:26:03.346675+0000","last_clean":"2024-04-26T20:26:03.346675+0000","last_became_active":"2024-04-26T20:25:56.985636+0000","last_became_peered":"2024-04-26T20:25:56.985636+0000","last_unstale":"2024-04-26T20:26:03.346675+0000","last_undegraded":"2024-04-26T20:26:03.346675+0000","last_fullsized":"2024-04-26T20:26:03.346675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351558+0000","last_change":"2024-04-26T20:26:02.079470+0000","last_active":"2024-04-26T20:26:03.351558+0000","last_peered":"2024-04-26T20:26:03.351558+0000","last_clean":"2024-04-26T20:26:03.351558+0000","last_became_active":"2024-04-26T20:26:02.078853+0000","last_became_peered":"2024-04-26T20:26:02.078853+0000","last_unstale":"2024-04-26T20:26:03.351558+0000","last_undegraded":"2024-04-26T20:26:03.351558+0000","last_fullsized":"2024-04-26T20:26:03.351558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093119+0000","last_change":"2024-04-26T20:25:58.006907+0000","last_active":"2024-04-26T20:26:03.093119+0000","last_peered":"2024-04-26T20:26:03.093119+0000","last_clean":"2024-04-26T20:26:03.093119+0000","last_became_active":"2024-04-26T20:25:58.006370+0000","last_became_peered":"2024-04-26T20:25:58.006370+0000","last_unstale":"2024-04-26T20:26:03.093119+0000","last_undegraded":"2024-04-26T20:26:03.093119+0000","last_fullsized":"2024-04-26T20:26:03.093119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093249+0000","last_change":"2024-04-26T20:25:56.982952+0000","last_active":"2024-04-26T20:26:03.093249+0000","last_peered":"2024-04-26T20:26:03.093249+0000","last_clean":"2024-04-26T20:26:03.093249+0000","last_became_active":"2024-04-26T20:25:56.982240+0000","last_became_peered":"2024-04-26T20:25:56.982240+0000","last_unstale":"2024-04-26T20:26:03.093249+0000","last_undegraded":"2024-04-26T20:26:03.093249+0000","last_fullsized":"2024-04-26T20:26:03.093249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349655+0000","last_change":"2024-04-26T20:25:56.982228+0000","last_active":"2024-04-26T20:26:03.349655+0000","last_peered":"2024-04-26T20:26:03.349655+0000","last_clean":"2024-04-26T20:26:03.349655+0000","last_became_active":"2024-04-26T20:25:56.981083+0000","last_became_peered":"2024-04-26T20:25:56.981083+0000","last_unstale":"2024-04-26T20:26:03.349655+0000","last_undegraded":"2024-04-26T20:26:03.349655+0000","last_fullsized":"2024-04-26T20:26:03.349655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346695+0000","last_change":"2024-04-26T20:25:58.001685+0000","last_active":"2024-04-26T20:26:03.346695+0000","last_peered":"2024-04-26T20:26:03.346695+0000","last_clean":"2024-04-26T20:26:03.346695+0000","last_became_active":"2024-04-26T20:25:58.001160+0000","last_became_peered":"2024-04-26T20:25:58.001160+0000","last_unstale":"2024-04-26T20:26:03.346695+0000","last_undegraded":"2024-04-26T20:26:03.346695+0000","last_fullsized":"2024-04-26T20:26:03.346695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048776+0000","last_change":"2024-04-26T20:26:02.078797+0000","last_active":"2024-04-26T20:26:03.048776+0000","last_peered":"2024-04-26T20:26:03.048776+0000","last_clean":"2024-04-26T20:26:03.048776+0000","last_became_active":"2024-04-26T20:26:02.078444+0000","last_became_peered":"2024-04-26T20:26:02.078444+0000","last_unstale":"2024-04-26T20:26:03.048776+0000","last_undegraded":"2024-04-26T20:26:03.048776+0000","last_fullsized":"2024-04-26T20:26:03.048776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:16.872808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093946+0000","last_change":"2024-04-26T20:25:56.986739+0000","last_active":"2024-04-26T20:26:03.093946+0000","last_peered":"2024-04-26T20:26:03.093946+0000","last_clean":"2024-04-26T20:26:03.093946+0000","last_became_active":"2024-04-26T20:25:56.986399+0000","last_became_peered":"2024-04-26T20:25:56.986399+0000","last_unstale":"2024-04-26T20:26:03.093946+0000","last_undegraded":"2024-04-26T20:26:03.093946+0000","last_fullsized":"2024-04-26T20:26:03.093946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050966+0000","last_change":"2024-04-26T20:25:57.999363+0000","last_active":"2024-04-26T20:26:03.050966+0000","last_peered":"2024-04-26T20:26:03.050966+0000","last_clean":"2024-04-26T20:26:03.050966+0000","last_became_active":"2024-04-26T20:25:57.995297+0000","last_became_peered":"2024-04-26T20:25:57.995297+0000","last_unstale":"2024-04-26T20:26:03.050966+0000","last_undegraded":"2024-04-26T20:26:03.050966+0000","last_fullsized":"2024-04-26T20:26:03.050966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.671294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094404+0000","last_change":"2024-04-26T20:26:02.075522+0000","last_active":"2024-04-26T20:26:03.094404+0000","last_peered":"2024-04-26T20:26:03.094404+0000","last_clean":"2024-04-26T20:26:03.094404+0000","last_became_active":"2024-04-26T20:26:02.075089+0000","last_became_peered":"2024-04-26T20:26:02.075089+0000","last_unstale":"2024-04-26T20:26:03.094404+0000","last_undegraded":"2024-04-26T20:26:03.094404+0000","last_fullsized":"2024-04-26T20:26:03.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:23.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350468+0000","last_change":"2024-04-26T20:25:57.990722+0000","last_active":"2024-04-26T20:26:03.350468+0000","last_peered":"2024-04-26T20:26:03.350468+0000","last_clean":"2024-04-26T20:26:03.350468+0000","last_became_active":"2024-04-26T20:25:57.990382+0000","last_became_peered":"2024-04-26T20:25:57.990382+0000","last_unstale":"2024-04-26T20:26:03.350468+0000","last_undegraded":"2024-04-26T20:26:03.350468+0000","last_fullsized":"2024-04-26T20:26:03.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047754+0000","last_change":"2024-04-26T20:25:58.014226+0000","last_active":"2024-04-26T20:26:03.047754+0000","last_peered":"2024-04-26T20:26:03.047754+0000","last_clean":"2024-04-26T20:26:03.047754+0000","last_became_active":"2024-04-26T20:25:58.013319+0000","last_became_peered":"2024-04-26T20:25:58.013319+0000","last_unstale":"2024-04-26T20:26:03.047754+0000","last_undegraded":"2024-04-26T20:26:03.047754+0000","last_fullsized":"2024-04-26T20:26:03.047754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348155+0000","last_change":"2024-04-26T20:25:58.008173+0000","last_active":"2024-04-26T20:26:03.348155+0000","last_peered":"2024-04-26T20:26:03.348155+0000","last_clean":"2024-04-26T20:26:03.348155+0000","last_became_active":"2024-04-26T20:25:58.007377+0000","last_became_peered":"2024-04-26T20:25:58.007377+0000","last_unstale":"2024-04-26T20:26:03.348155+0000","last_undegraded":"2024-04-26T20:26:03.348155+0000","last_fullsized":"2024-04-26T20:26:03.348155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346409+0000","last_change":"2024-04-26T20:25:58.003184+0000","last_active":"2024-04-26T20:26:03.346409+0000","last_peered":"2024-04-26T20:26:03.346409+0000","last_clean":"2024-04-26T20:26:03.346409+0000","last_became_active":"2024-04-26T20:25:58.002849+0000","last_became_peered":"2024-04-26T20:25:58.002849+0000","last_unstale":"2024-04-26T20:26:03.346409+0000","last_undegraded":"2024-04-26T20:26:03.346409+0000","last_fullsized":"2024-04-26T20:26:03.346409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094328+0000","last_change":"2024-04-26T20:25:57.994419+0000","last_active":"2024-04-26T20:26:03.094328+0000","last_peered":"2024-04-26T20:26:03.094328+0000","last_clean":"2024-04-26T20:26:03.094328+0000","last_became_active":"2024-04-26T20:25:57.994077+0000","last_became_peered":"2024-04-26T20:25:57.994077+0000","last_unstale":"2024-04-26T20:26:03.094328+0000","last_undegraded":"2024-04-26T20:26:03.094328+0000","last_fullsized":"2024-04-26T20:26:03.094328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344289+0000","last_change":"2024-04-26T20:25:58.000446+0000","last_active":"2024-04-26T20:26:03.344289+0000","last_peered":"2024-04-26T20:26:03.344289+0000","last_clean":"2024-04-26T20:26:03.344289+0000","last_became_active":"2024-04-26T20:25:58.000166+0000","last_became_peered":"2024-04-26T20:25:58.000166+0000","last_unstale":"2024-04-26T20:26:03.344289+0000","last_undegraded":"2024-04-26T20:26:03.344289+0000","last_fullsized":"2024-04-26T20:26:03.344289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351690+0000","last_change":"2024-04-26T20:25:57.996288+0000","last_active":"2024-04-26T20:26:03.351690+0000","last_peered":"2024-04-26T20:26:03.351690+0000","last_clean":"2024-04-26T20:26:03.351690+0000","last_became_active":"2024-04-26T20:25:57.995978+0000","last_became_peered":"2024-04-26T20:25:57.995978+0000","last_unstale":"2024-04-26T20:26:03.351690+0000","last_undegraded":"2024-04-26T20:26:03.351690+0000","last_fullsized":"2024-04-26T20:26:03.351690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094950+0000","last_change":"2024-04-26T20:25:57.993394+0000","last_active":"2024-04-26T20:26:03.094950+0000","last_peered":"2024-04-26T20:26:03.094950+0000","last_clean":"2024-04-26T20:26:03.094950+0000","last_became_active":"2024-04-26T20:25:57.993023+0000","last_became_peered":"2024-04-26T20:25:57.993023+0000","last_unstale":"2024-04-26T20:26:03.094950+0000","last_undegraded":"2024-04-26T20:26:03.094950+0000","last_fullsized":"2024-04-26T20:26:03.094950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346415+0000","last_change":"2024-04-26T20:25:57.997685+0000","last_active":"2024-04-26T20:26:03.346415+0000","last_peered":"2024-04-26T20:26:03.346415+0000","last_clean":"2024-04-26T20:26:03.346415+0000","last_became_active":"2024-04-26T20:25:57.997422+0000","last_became_peered":"2024-04-26T20:25:57.997422+0000","last_unstale":"2024-04-26T20:26:03.346415+0000","last_undegraded":"2024-04-26T20:26:03.346415+0000","last_fullsized":"2024-04-26T20:26:03.346415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093095+0000","last_change":"2024-04-26T20:25:58.003864+0000","last_active":"2024-04-26T20:26:03.093095+0000","last_peered":"2024-04-26T20:26:03.093095+0000","last_clean":"2024-04-26T20:26:03.093095+0000","last_became_active":"2024-04-26T20:25:58.003553+0000","last_became_peered":"2024-04-26T20:25:58.003553+0000","last_unstale":"2024-04-26T20:26:03.093095+0000","last_undegraded":"2024-04-26T20:26:03.093095+0000","last_fullsized":"2024-04-26T20:26:03.093095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347138+0000","last_change":"2024-04-26T20:25:58.007689+0000","last_active":"2024-04-26T20:26:03.347138+0000","last_peered":"2024-04-26T20:26:03.347138+0000","last_clean":"2024-04-26T20:26:03.347138+0000","last_became_active":"2024-04-26T20:25:58.006743+0000","last_became_peered":"2024-04-26T20:25:58.006743+0000","last_unstale":"2024-04-26T20:26:03.347138+0000","last_undegraded":"2024-04-26T20:26:03.347138+0000","last_fullsized":"2024-04-26T20:26:03.347138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346030+0000","last_change":"2024-04-26T20:25:58.001516+0000","last_active":"2024-04-26T20:26:03.346030+0000","last_peered":"2024-04-26T20:26:03.346030+0000","last_clean":"2024-04-26T20:26:03.346030+0000","last_became_active":"2024-04-26T20:25:58.001241+0000","last_became_peered":"2024-04-26T20:25:58.001241+0000","last_unstale":"2024-04-26T20:26:03.346030+0000","last_undegraded":"2024-04-26T20:26:03.346030+0000","last_fullsized":"2024-04-26T20:26:03.346030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350860+0000","last_change":"2024-04-26T20:25:57.993316+0000","last_active":"2024-04-26T20:26:03.350860+0000","last_peered":"2024-04-26T20:26:03.350860+0000","last_clean":"2024-04-26T20:26:03.350860+0000","last_became_active":"2024-04-26T20:25:57.991497+0000","last_became_peered":"2024-04-26T20:25:57.991497+0000","last_unstale":"2024-04-26T20:26:03.350860+0000","last_undegraded":"2024-04-26T20:26:03.350860+0000","last_fullsized":"2024-04-26T20:26:03.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348333+0000","last_change":"2024-04-26T20:25:58.004031+0000","last_active":"2024-04-26T20:26:03.348333+0000","last_peered":"2024-04-26T20:26:03.348333+0000","last_clean":"2024-04-26T20:26:03.348333+0000","last_became_active":"2024-04-26T20:25:58.003728+0000","last_became_peered":"2024-04-26T20:25:58.003728+0000","last_unstale":"2024-04-26T20:26:03.348333+0000","last_undegraded":"2024-04-26T20:26:03.348333+0000","last_fullsized":"2024-04-26T20:26:03.348333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:28.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048584+0000","last_change":"2024-04-26T20:25:57.986712+0000","last_active":"2024-04-26T20:26:03.048584+0000","last_peered":"2024-04-26T20:26:03.048584+0000","last_clean":"2024-04-26T20:26:03.048584+0000","last_became_active":"2024-04-26T20:25:57.986233+0000","last_became_peered":"2024-04-26T20:25:57.986233+0000","last_unstale":"2024-04-26T20:26:03.048584+0000","last_undegraded":"2024-04-26T20:26:03.048584+0000","last_fullsized":"2024-04-26T20:26:03.048584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346524+0000","last_change":"2024-04-26T20:25:58.011317+0000","last_active":"2024-04-26T20:26:03.346524+0000","last_peered":"2024-04-26T20:26:03.346524+0000","last_clean":"2024-04-26T20:26:03.346524+0000","last_became_active":"2024-04-26T20:25:58.010881+0000","last_became_peered":"2024-04-26T20:25:58.010881+0000","last_unstale":"2024-04-26T20:26:03.346524+0000","last_undegraded":"2024-04-26T20:26:03.346524+0000","last_fullsized":"2024-04-26T20:26:03.346524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344914+0000","last_change":"2024-04-26T20:25:58.018419+0000","last_active":"2024-04-26T20:26:03.344914+0000","last_peered":"2024-04-26T20:26:03.344914+0000","last_clean":"2024-04-26T20:26:03.344914+0000","last_became_active":"2024-04-26T20:25:58.018134+0000","last_became_peered":"2024-04-26T20:25:58.018134+0000","last_unstale":"2024-04-26T20:26:03.344914+0000","last_undegraded":"2024-04-26T20:26:03.344914+0000","last_fullsized":"2024-04-26T20:26:03.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343902+0000","last_change":"2024-04-26T20:25:58.008263+0000","last_active":"2024-04-26T20:26:03.343902+0000","last_peered":"2024-04-26T20:26:03.343902+0000","last_clean":"2024-04-26T20:26:03.343902+0000","last_became_active":"2024-04-26T20:25:58.007268+0000","last_became_peered":"2024-04-26T20:25:58.007268+0000","last_unstale":"2024-04-26T20:26:03.343902+0000","last_undegraded":"2024-04-26T20:26:03.343902+0000","last_fullsized":"2024-04-26T20:26:03.343902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345391+0000","last_change":"2024-04-26T20:25:58.011832+0000","last_active":"2024-04-26T20:26:03.345391+0000","last_peered":"2024-04-26T20:26:03.345391+0000","last_clean":"2024-04-26T20:26:03.345391+0000","last_became_active":"2024-04-26T20:25:58.011007+0000","last_became_peered":"2024-04-26T20:25:58.011007+0000","last_unstale":"2024-04-26T20:26:03.345391+0000","last_undegraded":"2024-04-26T20:26:03.345391+0000","last_fullsized":"2024-04-26T20:26:03.345391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:38.909585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093783+0000","last_change":"2024-04-26T20:25:57.993780+0000","last_active":"2024-04-26T20:26:03.093783+0000","last_peered":"2024-04-26T20:26:03.093783+0000","last_clean":"2024-04-26T20:26:03.093783+0000","last_became_active":"2024-04-26T20:25:57.993444+0000","last_became_peered":"2024-04-26T20:25:57.993444+0000","last_unstale":"2024-04-26T20:26:03.093783+0000","last_undegraded":"2024-04-26T20:26:03.093783+0000","last_fullsized":"2024-04-26T20:26:03.093783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050252+0000","last_change":"2024-04-26T20:25:57.985098+0000","last_active":"2024-04-26T20:26:03.050252+0000","last_peered":"2024-04-26T20:26:03.050252+0000","last_clean":"2024-04-26T20:26:03.050252+0000","last_became_active":"2024-04-26T20:25:57.984739+0000","last_became_peered":"2024-04-26T20:25:57.984739+0000","last_unstale":"2024-04-26T20:26:03.050252+0000","last_undegraded":"2024-04-26T20:26:03.050252+0000","last_fullsized":"2024-04-26T20:26:03.050252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351223+0000","last_change":"2024-04-26T20:25:58.007442+0000","last_active":"2024-04-26T20:26:03.351223+0000","last_peered":"2024-04-26T20:26:03.351223+0000","last_clean":"2024-04-26T20:26:03.351223+0000","last_became_active":"2024-04-26T20:25:58.007150+0000","last_became_peered":"2024-04-26T20:25:58.007150+0000","last_unstale":"2024-04-26T20:26:03.351223+0000","last_undegraded":"2024-04-26T20:26:03.351223+0000","last_fullsized":"2024-04-26T20:26:03.351223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351686+0000","last_change":"2024-04-26T20:25:58.007696+0000","last_active":"2024-04-26T20:26:03.351686+0000","last_peered":"2024-04-26T20:26:03.351686+0000","last_clean":"2024-04-26T20:26:03.351686+0000","last_became_active":"2024-04-26T20:25:58.007415+0000","last_became_peered":"2024-04-26T20:25:58.007415+0000","last_unstale":"2024-04-26T20:26:03.351686+0000","last_undegraded":"2024-04-26T20:26:03.351686+0000","last_fullsized":"2024-04-26T20:26:03.351686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350018+0000","last_change":"2024-04-26T20:25:58.014108+0000","last_active":"2024-04-26T20:26:03.350018+0000","last_peered":"2024-04-26T20:26:03.350018+0000","last_clean":"2024-04-26T20:26:03.350018+0000","last_became_active":"2024-04-26T20:25:58.012072+0000","last_became_peered":"2024-04-26T20:25:58.012072+0000","last_unstale":"2024-04-26T20:26:03.350018+0000","last_undegraded":"2024-04-26T20:26:03.350018+0000","last_fullsized":"2024-04-26T20:26:03.350018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:10.380322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092736+0000","last_change":"2024-04-26T20:25:58.004587+0000","last_active":"2024-04-26T20:26:03.092736+0000","last_peered":"2024-04-26T20:26:03.092736+0000","last_clean":"2024-04-26T20:26:03.092736+0000","last_became_active":"2024-04-26T20:25:58.003068+0000","last_became_peered":"2024-04-26T20:25:58.003068+0000","last_unstale":"2024-04-26T20:26:03.092736+0000","last_undegraded":"2024-04-26T20:26:03.092736+0000","last_fullsized":"2024-04-26T20:26:03.092736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345668+0000","last_change":"2024-04-26T20:25:58.017953+0000","last_active":"2024-04-26T20:26:03.345668+0000","last_peered":"2024-04-26T20:26:03.345668+0000","last_clean":"2024-04-26T20:26:03.345668+0000","last_became_active":"2024-04-26T20:25:58.017321+0000","last_became_peered":"2024-04-26T20:25:58.017321+0000","last_unstale":"2024-04-26T20:26:03.345668+0000","last_undegraded":"2024-04-26T20:26:03.345668+0000","last_fullsized":"2024-04-26T20:26:03.345668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346846+0000","last_change":"2024-04-26T20:25:58.007941+0000","last_active":"2024-04-26T20:26:03.346846+0000","last_peered":"2024-04-26T20:26:03.346846+0000","last_clean":"2024-04-26T20:26:03.346846+0000","last_became_active":"2024-04-26T20:25:58.007043+0000","last_became_peered":"2024-04-26T20:25:58.007043+0000","last_unstale":"2024-04-26T20:26:03.346846+0000","last_undegraded":"2024-04-26T20:26:03.346846+0000","last_fullsized":"2024-04-26T20:26:03.346846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347334+0000","last_change":"2024-04-26T20:25:58.003034+0000","last_active":"2024-04-26T20:26:03.347334+0000","last_peered":"2024-04-26T20:26:03.347334+0000","last_clean":"2024-04-26T20:26:03.347334+0000","last_became_active":"2024-04-26T20:25:58.002792+0000","last_became_peered":"2024-04-26T20:25:58.002792+0000","last_unstale":"2024-04-26T20:26:03.347334+0000","last_undegraded":"2024-04-26T20:26:03.347334+0000","last_fullsized":"2024-04-26T20:26:03.347334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346106+0000","last_change":"2024-04-26T20:25:58.008698+0000","last_active":"2024-04-26T20:26:03.346106+0000","last_peered":"2024-04-26T20:26:03.346106+0000","last_clean":"2024-04-26T20:26:03.346106+0000","last_became_active":"2024-04-26T20:25:58.008279+0000","last_became_peered":"2024-04-26T20:25:58.008279+0000","last_unstale":"2024-04-26T20:26:03.346106+0000","last_undegraded":"2024-04-26T20:26:03.346106+0000","last_fullsized":"2024-04-26T20:26:03.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346355+0000","last_change":"2024-04-26T20:25:57.985508+0000","last_active":"2024-04-26T20:26:03.346355+0000","last_peered":"2024-04-26T20:26:03.346355+0000","last_clean":"2024-04-26T20:26:03.346355+0000","last_became_active":"2024-04-26T20:25:57.985227+0000","last_became_peered":"2024-04-26T20:25:57.985227+0000","last_unstale":"2024-04-26T20:26:03.346355+0000","last_undegraded":"2024-04-26T20:26:03.346355+0000","last_fullsized":"2024-04-26T20:26:03.346355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346661+0000","last_change":"2024-04-26T20:25:57.989552+0000","last_active":"2024-04-26T20:26:03.346661+0000","last_peered":"2024-04-26T20:26:03.346661+0000","last_clean":"2024-04-26T20:26:03.346661+0000","last_became_active":"2024-04-26T20:25:57.989141+0000","last_became_peered":"2024-04-26T20:25:57.989141+0000","last_unstale":"2024-04-26T20:26:03.346661+0000","last_undegraded":"2024-04-26T20:26:03.346661+0000","last_fullsized":"2024-04-26T20:26:03.346661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349524+0000","last_change":"2024-04-26T20:25:58.003504+0000","last_active":"2024-04-26T20:26:03.349524+0000","last_peered":"2024-04-26T20:26:03.349524+0000","last_clean":"2024-04-26T20:26:03.349524+0000","last_became_active":"2024-04-26T20:25:58.002880+0000","last_became_peered":"2024-04-26T20:25:58.002880+0000","last_unstale":"2024-04-26T20:26:03.349524+0000","last_undegraded":"2024-04-26T20:26:03.349524+0000","last_fullsized":"2024-04-26T20:26:03.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:47.751608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:26.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.876+0000 7f1f63600640 1 Processor -- start 2024-04-26T20:26:26.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.876+0000 7f1f63600640 1 -- start start 2024-04-26T20:26:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f63600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c145590 0x7f1f5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f63600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1f5c06f730 0x7f1f5c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f63600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5c0cf060 con 0x7f1f5c0b3490 2024-04-26T20:26:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f63600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c0d0100 con 0x7f1f5c145590 2024-04-26T20:26:26.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f63600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c0cf340 con 0x7f1f5c06f730 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c145590 0x7f1f5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c145590 0x7f1f5c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54516/0 (socket says 172.21.15.43:54516) 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 -- 172.21.15.43:0/1010607507 learned_addr learned my addr 172.21.15.43:0/1010607507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f61600640 1 --2- 172.21.15.43:0/1010607507 >> v2:172.21.15.164:3300/0 conn(0x7f1f5c06f730 0x7f1f5c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 -- 172.21.15.43:0/1010607507 >> v2:172.21.15.164:3300/0 conn(0x7f1f5c06f730 msgr2=0x7f1f5c071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 --2- 172.21.15.43:0/1010607507 >> v2:172.21.15.164:3300/0 conn(0x7f1f5c06f730 0x7f1f5c071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.878+0000 7f1f60c00640 1 -- 172.21.15.43:0/1010607507 >> v1:172.21.15.164:6789/0 conn(0x7f1f5c0b3490 legacy=0x7f1f5c06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.879+0000 7f1f60c00640 1 -- 172.21.15.43:0/1010607507 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c0cf620 con 0x7f1f5c145590 2024-04-26T20:26:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f60c00640 1 --2- 172.21.15.43:0/1010607507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0ce780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0d1b70 tx=0x7f1f4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b727067dea4871 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f5b400640 1 -- 172.21.15.43:0/1010607507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48037070 con 0x7f1f5c145590 2024-04-26T20:26:26.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f5b400640 1 -- 172.21.15.43:0/1010607507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f48032440 con 0x7f1f5c145590 2024-04-26T20:26:26.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f5b400640 1 -- 172.21.15.43:0/1010607507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48032810 con 0x7f1f5c145590 2024-04-26T20:26:26.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f63600640 1 -- 172.21.15.43:0/1010607507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 msgr2=0x7f1f5c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.880+0000 7f1f63600640 1 --2- 172.21.15.43:0/1010607507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0ce780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0d1b70 tx=0x7f1f4802a230 comp rx=0 tx=0).stop 2024-04-26T20:26:26.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 -- 172.21.15.43:0/1010607507 shutdown_connections 2024-04-26T20:26:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 --2- 172.21.15.43:0/1010607507 >> v2:172.21.15.164:3300/0 conn(0x7f1f5c06f730 0x7f1f5c071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 --2- 172.21.15.43:0/1010607507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0ce780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 -- 172.21.15.43:0/1010607507 >> 172.21.15.43:0/1010607507 conn(0x7f1f5c0b2900 msgr2=0x7f1f5c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 -- 172.21.15.43:0/1010607507 shutdown_connections 2024-04-26T20:26:26.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.881+0000 7f1f63600640 1 -- 172.21.15.43:0/1010607507 wait complete. 2024-04-26T20:26:26.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.882+0000 7f1f63600640 1 Processor -- start 2024-04-26T20:26:26.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 -- start start 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c06f730 0x7f1f5c1411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f5c0b3490 0x7f1f5c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c145970 con 0x7f1f5c145590 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0cfbe0 con 0x7f1f5c06f730 2024-04-26T20:26:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.883+0000 7f1f63600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5c0cecc0 con 0x7f1f5c0b3490 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.884+0000 7f1f61600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.884+0000 7f1f61600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54524/0 (socket says 172.21.15.43:54524) 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.884+0000 7f1f61600640 1 -- 172.21.15.43:0/775198196 learned_addr learned my addr 172.21.15.43:0/775198196 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.884+0000 7f1f60c00640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c06f730 0x7f1f5c1411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.884+0000 7f1f5be00640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f5c0b3490 0x7f1f5c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 -- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f5c0b3490 msgr2=0x7f1f5c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f5c0b3490 0x7f1f5c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 -- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c06f730 msgr2=0x7f1f5c1411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c06f730 0x7f1f5c1411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 -- 172.21.15.43:0/775198196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c142860 con 0x7f1f5c145590 2024-04-26T20:26:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f61600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f54007e40 tx=0x7f1f5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54014620 con 0x7f1f5c145590 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f54019070 con 0x7f1f5c145590 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54014a20 con 0x7f1f5c145590 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c142b40 con 0x7f1f5c145590 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.885+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c143100 con 0x7f1f5c145590 2024-04-26T20:26:26.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.887+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5c069350 con 0x7f1f5c145590 2024-04-26T20:26:26.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.888+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f54014bc0 con 0x7f1f5c145590 2024-04-26T20:26:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.889+0000 7f1f58c00640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 0x7f1f3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.889+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f5408c7e0 con 0x7f1f5c145590 2024-04-26T20:26:26.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.889+0000 7f1f60c00640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 0x7f1f3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:26.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.890+0000 7f1f60c00640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 0x7f1f3c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f48034000 tx=0x7f1f4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:26.894+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f5405a7a0 con 0x7f1f5c145590 2024-04-26T20:26:27.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:27.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:27.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.093+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f5c06e350 con 0x7f1f3c06c8f0 2024-04-26T20:26:27.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.102+0000 7f1f58c00640 1 -- 172.21.15.43:0/775198196 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7f1f5c06e350 con 0x7f1f3c06c8f0 2024-04-26T20:26:27.107 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:27.109 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:26:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 msgr2=0x7f1f3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 0x7f1f3c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f48034000 tx=0x7f1f4802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 msgr2=0x7f1f5c0722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f54007e40 tx=0x7f1f5400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 shutdown_connections 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1f3c06c8f0 0x7f1f3c06edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1f5c0b3490 0x7f1f5c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f5c145590 0x7f1f5c0722f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 --2- 172.21.15.43:0/775198196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1f5c06f730 0x7f1f5c1411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 >> 172.21.15.43:0/775198196 conn(0x7f1f5c0b2900 msgr2=0x7f1f5c0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 shutdown_connections 2024-04-26T20:26:27.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.107+0000 7f1f63600640 1 -- 172.21.15.43:0/775198196 wait complete. 2024-04-26T20:26:27.128 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T20:26:26.867472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227728,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746992,"statfs":{"total":773094113280,"available":772860919808,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005070"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347756+0000","last_change":"2024-04-26T20:25:56.984031+0000","last_active":"2024-04-26T20:26:03.347756+0000","last_peered":"2024-04-26T20:26:03.347756+0000","last_clean":"2024-04-26T20:26:03.347756+0000","last_became_active":"2024-04-26T20:25:56.983640+0000","last_became_peered":"2024-04-26T20:25:56.983640+0000","last_unstale":"2024-04-26T20:26:03.347756+0000","last_undegraded":"2024-04-26T20:26:03.347756+0000","last_fullsized":"2024-04-26T20:26:03.347756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:04.709694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094688+0000","last_change":"2024-04-26T20:25:57.999359+0000","last_active":"2024-04-26T20:26:03.094688+0000","last_peered":"2024-04-26T20:26:03.094688+0000","last_clean":"2024-04-26T20:26:03.094688+0000","last_became_active":"2024-04-26T20:25:57.999124+0000","last_became_peered":"2024-04-26T20:25:57.999124+0000","last_unstale":"2024-04-26T20:26:03.094688+0000","last_undegraded":"2024-04-26T20:26:03.094688+0000","last_fullsized":"2024-04-26T20:26:03.094688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092925+0000","last_change":"2024-04-26T20:26:02.417831+0000","last_active":"2024-04-26T20:26:03.092925+0000","last_peered":"2024-04-26T20:26:03.092925+0000","last_clean":"2024-04-26T20:26:03.092925+0000","last_became_active":"2024-04-26T20:26:02.417256+0000","last_became_peered":"2024-04-26T20:26:02.417256+0000","last_unstale":"2024-04-26T20:26:03.092925+0000","last_undegraded":"2024-04-26T20:26:03.092925+0000","last_fullsized":"2024-04-26T20:26:03.092925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094948+0000","last_change":"2024-04-26T20:25:56.981647+0000","last_active":"2024-04-26T20:26:03.094948+0000","last_peered":"2024-04-26T20:26:03.094948+0000","last_clean":"2024-04-26T20:26:03.094948+0000","last_became_active":"2024-04-26T20:25:56.980997+0000","last_became_peered":"2024-04-26T20:25:56.980997+0000","last_unstale":"2024-04-26T20:26:03.094948+0000","last_undegraded":"2024-04-26T20:26:03.094948+0000","last_fullsized":"2024-04-26T20:26:03.094948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347273+0000","last_change":"2024-04-26T20:25:58.006366+0000","last_active":"2024-04-26T20:26:03.347273+0000","last_peered":"2024-04-26T20:26:03.347273+0000","last_clean":"2024-04-26T20:26:03.347273+0000","last_became_active":"2024-04-26T20:25:58.005106+0000","last_became_peered":"2024-04-26T20:25:58.005106+0000","last_unstale":"2024-04-26T20:26:03.347273+0000","last_undegraded":"2024-04-26T20:26:03.347273+0000","last_fullsized":"2024-04-26T20:26:03.347273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050650+0000","last_change":"2024-04-26T20:26:02.077388+0000","last_active":"2024-04-26T20:26:03.050650+0000","last_peered":"2024-04-26T20:26:03.050650+0000","last_clean":"2024-04-26T20:26:03.050650+0000","last_became_active":"2024-04-26T20:26:02.076618+0000","last_became_peered":"2024-04-26T20:26:02.076618+0000","last_unstale":"2024-04-26T20:26:03.050650+0000","last_undegraded":"2024-04-26T20:26:03.050650+0000","last_fullsized":"2024-04-26T20:26:03.050650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:59.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348208+0000","last_change":"2024-04-26T20:25:56.993261+0000","last_active":"2024-04-26T20:26:03.348208+0000","last_peered":"2024-04-26T20:26:03.348208+0000","last_clean":"2024-04-26T20:26:03.348208+0000","last_became_active":"2024-04-26T20:25:56.992657+0000","last_became_peered":"2024-04-26T20:25:56.992657+0000","last_unstale":"2024-04-26T20:26:03.348208+0000","last_undegraded":"2024-04-26T20:26:03.348208+0000","last_fullsized":"2024-04-26T20:26:03.348208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346205+0000","last_change":"2024-04-26T20:25:58.008848+0000","last_active":"2024-04-26T20:26:03.346205+0000","last_peered":"2024-04-26T20:26:03.346205+0000","last_clean":"2024-04-26T20:26:03.346205+0000","last_became_active":"2024-04-26T20:25:58.008316+0000","last_became_peered":"2024-04-26T20:25:58.008316+0000","last_unstale":"2024-04-26T20:26:03.346205+0000","last_undegraded":"2024-04-26T20:26:03.346205+0000","last_fullsized":"2024-04-26T20:26:03.346205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051568+0000","last_change":"2024-04-26T20:26:02.080666+0000","last_active":"2024-04-26T20:26:03.051568+0000","last_peered":"2024-04-26T20:26:03.051568+0000","last_clean":"2024-04-26T20:26:03.051568+0000","last_became_active":"2024-04-26T20:26:02.080286+0000","last_became_peered":"2024-04-26T20:26:02.080286+0000","last_unstale":"2024-04-26T20:26:03.051568+0000","last_undegraded":"2024-04-26T20:26:03.051568+0000","last_fullsized":"2024-04-26T20:26:03.051568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347297+0000","last_change":"2024-04-26T20:25:56.993036+0000","last_active":"2024-04-26T20:26:03.347297+0000","last_peered":"2024-04-26T20:26:03.347297+0000","last_clean":"2024-04-26T20:26:03.347297+0000","last_became_active":"2024-04-26T20:25:56.992712+0000","last_became_peered":"2024-04-26T20:25:56.992712+0000","last_unstale":"2024-04-26T20:26:03.347297+0000","last_undegraded":"2024-04-26T20:26:03.347297+0000","last_fullsized":"2024-04-26T20:26:03.347297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:11.781113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347210+0000","last_change":"2024-04-26T20:25:58.005509+0000","last_active":"2024-04-26T20:26:03.347210+0000","last_peered":"2024-04-26T20:26:03.347210+0000","last_clean":"2024-04-26T20:26:03.347210+0000","last_became_active":"2024-04-26T20:25:58.005266+0000","last_became_peered":"2024-04-26T20:25:58.005266+0000","last_unstale":"2024-04-26T20:26:03.347210+0000","last_undegraded":"2024-04-26T20:26:03.347210+0000","last_fullsized":"2024-04-26T20:26:03.347210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050541+0000","last_change":"2024-04-26T20:26:02.076146+0000","last_active":"2024-04-26T20:26:03.050541+0000","last_peered":"2024-04-26T20:26:03.050541+0000","last_clean":"2024-04-26T20:26:03.050541+0000","last_became_active":"2024-04-26T20:26:02.075597+0000","last_became_peered":"2024-04-26T20:26:02.075597+0000","last_unstale":"2024-04-26T20:26:03.050541+0000","last_undegraded":"2024-04-26T20:26:03.050541+0000","last_fullsized":"2024-04-26T20:26:03.050541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:49.620083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349354+0000","last_change":"2024-04-26T20:25:57.995411+0000","last_active":"2024-04-26T20:26:03.349354+0000","last_peered":"2024-04-26T20:26:03.349354+0000","last_clean":"2024-04-26T20:26:03.349354+0000","last_became_active":"2024-04-26T20:25:57.993865+0000","last_became_peered":"2024-04-26T20:25:57.993865+0000","last_unstale":"2024-04-26T20:26:03.349354+0000","last_undegraded":"2024-04-26T20:26:03.349354+0000","last_fullsized":"2024-04-26T20:26:03.349354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347802+0000","last_change":"2024-04-26T20:25:56.986121+0000","last_active":"2024-04-26T20:26:03.347802+0000","last_peered":"2024-04-26T20:26:03.347802+0000","last_clean":"2024-04-26T20:26:03.347802+0000","last_became_active":"2024-04-26T20:25:56.985836+0000","last_became_peered":"2024-04-26T20:25:56.985836+0000","last_unstale":"2024-04-26T20:26:03.347802+0000","last_undegraded":"2024-04-26T20:26:03.347802+0000","last_fullsized":"2024-04-26T20:26:03.347802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:01.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049987+0000","last_change":"2024-04-26T20:26:02.079776+0000","last_active":"2024-04-26T20:26:03.049987+0000","last_peered":"2024-04-26T20:26:03.049987+0000","last_clean":"2024-04-26T20:26:03.049987+0000","last_became_active":"2024-04-26T20:26:02.079355+0000","last_became_peered":"2024-04-26T20:26:02.079355+0000","last_unstale":"2024-04-26T20:26:03.049987+0000","last_undegraded":"2024-04-26T20:26:03.049987+0000","last_fullsized":"2024-04-26T20:26:03.049987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:32.933389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093856+0000","last_change":"2024-04-26T20:25:57.990924+0000","last_active":"2024-04-26T20:26:03.093856+0000","last_peered":"2024-04-26T20:26:03.093856+0000","last_clean":"2024-04-26T20:26:03.093856+0000","last_became_active":"2024-04-26T20:25:57.990615+0000","last_became_peered":"2024-04-26T20:25:57.990615+0000","last_unstale":"2024-04-26T20:26:03.093856+0000","last_undegraded":"2024-04-26T20:26:03.093856+0000","last_fullsized":"2024-04-26T20:26:03.093856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345060+0000","last_change":"2024-04-26T20:25:56.985786+0000","last_active":"2024-04-26T20:26:03.345060+0000","last_peered":"2024-04-26T20:26:03.345060+0000","last_clean":"2024-04-26T20:26:03.345060+0000","last_became_active":"2024-04-26T20:25:56.981498+0000","last_became_peered":"2024-04-26T20:25:56.981498+0000","last_unstale":"2024-04-26T20:26:03.345060+0000","last_undegraded":"2024-04-26T20:26:03.345060+0000","last_fullsized":"2024-04-26T20:26:03.345060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049606+0000","last_change":"2024-04-26T20:26:02.075573+0000","last_active":"2024-04-26T20:26:03.049606+0000","last_peered":"2024-04-26T20:26:03.049606+0000","last_clean":"2024-04-26T20:26:03.049606+0000","last_became_active":"2024-04-26T20:26:02.075012+0000","last_became_peered":"2024-04-26T20:26:02.075012+0000","last_unstale":"2024-04-26T20:26:03.049606+0000","last_undegraded":"2024-04-26T20:26:03.049606+0000","last_fullsized":"2024-04-26T20:26:03.049606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.401666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093255+0000","last_change":"2024-04-26T20:25:58.011840+0000","last_active":"2024-04-26T20:26:03.093255+0000","last_peered":"2024-04-26T20:26:03.093255+0000","last_clean":"2024-04-26T20:26:03.093255+0000","last_became_active":"2024-04-26T20:25:58.011346+0000","last_became_peered":"2024-04-26T20:25:58.011346+0000","last_unstale":"2024-04-26T20:26:03.093255+0000","last_undegraded":"2024-04-26T20:26:03.093255+0000","last_fullsized":"2024-04-26T20:26:03.093255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344179+0000","last_change":"2024-04-26T20:25:56.986011+0000","last_active":"2024-04-26T20:26:03.344179+0000","last_peered":"2024-04-26T20:26:03.344179+0000","last_clean":"2024-04-26T20:26:03.344179+0000","last_became_active":"2024-04-26T20:25:56.985728+0000","last_became_peered":"2024-04-26T20:25:56.985728+0000","last_unstale":"2024-04-26T20:26:03.344179+0000","last_undegraded":"2024-04-26T20:26:03.344179+0000","last_fullsized":"2024-04-26T20:26:03.344179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347438+0000","last_change":"2024-04-26T20:26:02.401041+0000","last_active":"2024-04-26T20:26:03.347438+0000","last_peered":"2024-04-26T20:26:03.347438+0000","last_clean":"2024-04-26T20:26:03.347438+0000","last_became_active":"2024-04-26T20:26:02.400108+0000","last_became_peered":"2024-04-26T20:26:02.400108+0000","last_unstale":"2024-04-26T20:26:03.347438+0000","last_undegraded":"2024-04-26T20:26:03.347438+0000","last_fullsized":"2024-04-26T20:26:03.347438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049644+0000","last_change":"2024-04-26T20:25:57.985787+0000","last_active":"2024-04-26T20:26:03.049644+0000","last_peered":"2024-04-26T20:26:03.049644+0000","last_clean":"2024-04-26T20:26:03.049644+0000","last_became_active":"2024-04-26T20:25:57.985432+0000","last_became_peered":"2024-04-26T20:25:57.985432+0000","last_unstale":"2024-04-26T20:26:03.049644+0000","last_undegraded":"2024-04-26T20:26:03.049644+0000","last_fullsized":"2024-04-26T20:26:03.049644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:37.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345394+0000","last_change":"2024-04-26T20:25:56.983944+0000","last_active":"2024-04-26T20:26:03.345394+0000","last_peered":"2024-04-26T20:26:03.345394+0000","last_clean":"2024-04-26T20:26:03.345394+0000","last_became_active":"2024-04-26T20:25:56.983642+0000","last_became_peered":"2024-04-26T20:25:56.983642+0000","last_unstale":"2024-04-26T20:26:03.345394+0000","last_undegraded":"2024-04-26T20:26:03.345394+0000","last_fullsized":"2024-04-26T20:26:03.345394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:12.976926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050080+0000","last_change":"2024-04-26T20:26:02.082133+0000","last_active":"2024-04-26T20:26:03.050080+0000","last_peered":"2024-04-26T20:26:03.050080+0000","last_clean":"2024-04-26T20:26:03.050080+0000","last_became_active":"2024-04-26T20:26:02.081547+0000","last_became_peered":"2024-04-26T20:26:02.081547+0000","last_unstale":"2024-04-26T20:26:03.050080+0000","last_undegraded":"2024-04-26T20:26:03.050080+0000","last_fullsized":"2024-04-26T20:26:03.050080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:39.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347003+0000","last_change":"2024-04-26T20:25:57.989909+0000","last_active":"2024-04-26T20:26:03.347003+0000","last_peered":"2024-04-26T20:26:03.347003+0000","last_clean":"2024-04-26T20:26:03.347003+0000","last_became_active":"2024-04-26T20:25:57.989626+0000","last_became_peered":"2024-04-26T20:25:57.989626+0000","last_unstale":"2024-04-26T20:26:03.347003+0000","last_undegraded":"2024-04-26T20:26:03.347003+0000","last_fullsized":"2024-04-26T20:26:03.347003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351956+0000","last_change":"2024-04-26T20:25:56.985557+0000","last_active":"2024-04-26T20:26:03.351956+0000","last_peered":"2024-04-26T20:26:03.351956+0000","last_clean":"2024-04-26T20:26:03.351956+0000","last_became_active":"2024-04-26T20:25:56.985217+0000","last_became_peered":"2024-04-26T20:25:56.985217+0000","last_unstale":"2024-04-26T20:26:03.351956+0000","last_undegraded":"2024-04-26T20:26:03.351956+0000","last_fullsized":"2024-04-26T20:26:03.351956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050981+0000","last_change":"2024-04-26T20:26:02.405217+0000","last_active":"2024-04-26T20:26:03.050981+0000","last_peered":"2024-04-26T20:26:03.050981+0000","last_clean":"2024-04-26T20:26:03.050981+0000","last_became_active":"2024-04-26T20:26:02.404124+0000","last_became_peered":"2024-04-26T20:26:02.404124+0000","last_unstale":"2024-04-26T20:26:03.050981+0000","last_undegraded":"2024-04-26T20:26:03.050981+0000","last_fullsized":"2024-04-26T20:26:03.050981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:29.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095109+0000","last_change":"2024-04-26T20:25:57.999075+0000","last_active":"2024-04-26T20:26:03.095109+0000","last_peered":"2024-04-26T20:26:03.095109+0000","last_clean":"2024-04-26T20:26:03.095109+0000","last_became_active":"2024-04-26T20:25:57.998666+0000","last_became_peered":"2024-04-26T20:25:57.998666+0000","last_unstale":"2024-04-26T20:26:03.095109+0000","last_undegraded":"2024-04-26T20:26:03.095109+0000","last_fullsized":"2024-04-26T20:26:03.095109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346736+0000","last_change":"2024-04-26T20:25:56.977150+0000","last_active":"2024-04-26T20:26:03.346736+0000","last_peered":"2024-04-26T20:26:03.346736+0000","last_clean":"2024-04-26T20:26:03.346736+0000","last_became_active":"2024-04-26T20:25:56.976895+0000","last_became_peered":"2024-04-26T20:25:56.976895+0000","last_unstale":"2024-04-26T20:26:03.346736+0000","last_undegraded":"2024-04-26T20:26:03.346736+0000","last_fullsized":"2024-04-26T20:26:03.346736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093148+0000","last_change":"2024-04-26T20:26:02.400496+0000","last_active":"2024-04-26T20:26:03.093148+0000","last_peered":"2024-04-26T20:26:03.093148+0000","last_clean":"2024-04-26T20:26:03.093148+0000","last_became_active":"2024-04-26T20:26:02.400002+0000","last_became_peered":"2024-04-26T20:26:02.400002+0000","last_unstale":"2024-04-26T20:26:03.093148+0000","last_undegraded":"2024-04-26T20:26:03.093148+0000","last_fullsized":"2024-04-26T20:26:03.093148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345843+0000","last_change":"2024-04-26T20:25:58.009127+0000","last_active":"2024-04-26T20:26:03.345843+0000","last_peered":"2024-04-26T20:26:03.345843+0000","last_clean":"2024-04-26T20:26:03.345843+0000","last_became_active":"2024-04-26T20:25:58.006542+0000","last_became_peered":"2024-04-26T20:25:58.006542+0000","last_unstale":"2024-04-26T20:26:03.345843+0000","last_undegraded":"2024-04-26T20:26:03.345843+0000","last_fullsized":"2024-04-26T20:26:03.345843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348511+0000","last_change":"2024-04-26T20:25:56.992958+0000","last_active":"2024-04-26T20:26:03.348511+0000","last_peered":"2024-04-26T20:26:03.348511+0000","last_clean":"2024-04-26T20:26:03.348511+0000","last_became_active":"2024-04-26T20:25:56.992326+0000","last_became_peered":"2024-04-26T20:25:56.992326+0000","last_unstale":"2024-04-26T20:26:03.348511+0000","last_undegraded":"2024-04-26T20:26:03.348511+0000","last_fullsized":"2024-04-26T20:26:03.348511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346294+0000","last_change":"2024-04-26T20:26:02.406897+0000","last_active":"2024-04-26T20:26:03.346294+0000","last_peered":"2024-04-26T20:26:03.346294+0000","last_clean":"2024-04-26T20:26:03.346294+0000","last_became_active":"2024-04-26T20:26:02.402553+0000","last_became_peered":"2024-04-26T20:26:02.402553+0000","last_unstale":"2024-04-26T20:26:03.346294+0000","last_undegraded":"2024-04-26T20:26:03.346294+0000","last_fullsized":"2024-04-26T20:26:03.346294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347458+0000","last_change":"2024-04-26T20:25:58.003889+0000","last_active":"2024-04-26T20:26:03.347458+0000","last_peered":"2024-04-26T20:26:03.347458+0000","last_clean":"2024-04-26T20:26:03.347458+0000","last_became_active":"2024-04-26T20:25:58.003578+0000","last_became_peered":"2024-04-26T20:25:58.003578+0000","last_unstale":"2024-04-26T20:26:03.347458+0000","last_undegraded":"2024-04-26T20:26:03.347458+0000","last_fullsized":"2024-04-26T20:26:03.347458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346054+0000","last_change":"2024-04-26T20:25:56.975939+0000","last_active":"2024-04-26T20:26:03.346054+0000","last_peered":"2024-04-26T20:26:03.346054+0000","last_clean":"2024-04-26T20:26:03.346054+0000","last_became_active":"2024-04-26T20:25:56.975669+0000","last_became_peered":"2024-04-26T20:25:56.975669+0000","last_unstale":"2024-04-26T20:26:03.346054+0000","last_undegraded":"2024-04-26T20:26:03.346054+0000","last_fullsized":"2024-04-26T20:26:03.346054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344705+0000","last_change":"2024-04-26T20:26:02.079073+0000","last_active":"2024-04-26T20:26:03.344705+0000","last_peered":"2024-04-26T20:26:03.344705+0000","last_clean":"2024-04-26T20:26:03.344705+0000","last_became_active":"2024-04-26T20:26:02.074779+0000","last_became_peered":"2024-04-26T20:26:02.074779+0000","last_unstale":"2024-04-26T20:26:03.344705+0000","last_undegraded":"2024-04-26T20:26:03.344705+0000","last_fullsized":"2024-04-26T20:26:03.344705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051836+0000","last_change":"2024-04-26T20:25:57.988308+0000","last_active":"2024-04-26T20:26:03.051836+0000","last_peered":"2024-04-26T20:26:03.051836+0000","last_clean":"2024-04-26T20:26:03.051836+0000","last_became_active":"2024-04-26T20:25:57.984254+0000","last_became_peered":"2024-04-26T20:25:57.984254+0000","last_unstale":"2024-04-26T20:26:03.051836+0000","last_undegraded":"2024-04-26T20:26:03.051836+0000","last_fullsized":"2024-04-26T20:26:03.051836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:26.721241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348724+0000","last_change":"2024-04-26T20:25:56.980841+0000","last_active":"2024-04-26T20:26:03.348724+0000","last_peered":"2024-04-26T20:26:03.348724+0000","last_clean":"2024-04-26T20:26:03.348724+0000","last_became_active":"2024-04-26T20:25:56.980215+0000","last_became_peered":"2024-04-26T20:25:56.980215+0000","last_unstale":"2024-04-26T20:26:03.348724+0000","last_undegraded":"2024-04-26T20:26:03.348724+0000","last_fullsized":"2024-04-26T20:26:03.348724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345720+0000","last_change":"2024-04-26T20:26:02.078478+0000","last_active":"2024-04-26T20:26:03.345720+0000","last_peered":"2024-04-26T20:26:03.345720+0000","last_clean":"2024-04-26T20:26:03.345720+0000","last_became_active":"2024-04-26T20:26:02.078121+0000","last_became_peered":"2024-04-26T20:26:02.078121+0000","last_unstale":"2024-04-26T20:26:03.345720+0000","last_undegraded":"2024-04-26T20:26:03.345720+0000","last_fullsized":"2024-04-26T20:26:03.345720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343920+0000","last_change":"2024-04-26T20:25:58.001121+0000","last_active":"2024-04-26T20:26:03.343920+0000","last_peered":"2024-04-26T20:26:03.343920+0000","last_clean":"2024-04-26T20:26:03.343920+0000","last_became_active":"2024-04-26T20:25:58.000412+0000","last_became_peered":"2024-04-26T20:25:58.000412+0000","last_unstale":"2024-04-26T20:26:03.343920+0000","last_undegraded":"2024-04-26T20:26:03.343920+0000","last_fullsized":"2024-04-26T20:26:03.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048086+0000","last_change":"2024-04-26T20:25:56.972300+0000","last_active":"2024-04-26T20:26:03.048086+0000","last_peered":"2024-04-26T20:26:03.048086+0000","last_clean":"2024-04-26T20:26:03.048086+0000","last_became_active":"2024-04-26T20:25:56.972020+0000","last_became_peered":"2024-04-26T20:25:56.972020+0000","last_unstale":"2024-04-26T20:26:03.048086+0000","last_undegraded":"2024-04-26T20:26:03.048086+0000","last_fullsized":"2024-04-26T20:26:03.048086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347521+0000","last_change":"2024-04-26T20:26:02.077923+0000","last_active":"2024-04-26T20:26:03.347521+0000","last_peered":"2024-04-26T20:26:03.347521+0000","last_clean":"2024-04-26T20:26:03.347521+0000","last_became_active":"2024-04-26T20:26:02.077483+0000","last_became_peered":"2024-04-26T20:26:02.077483+0000","last_unstale":"2024-04-26T20:26:03.347521+0000","last_undegraded":"2024-04-26T20:26:03.347521+0000","last_fullsized":"2024-04-26T20:26:03.347521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344678+0000","last_change":"2024-04-26T20:25:57.984633+0000","last_active":"2024-04-26T20:26:03.344678+0000","last_peered":"2024-04-26T20:26:03.344678+0000","last_clean":"2024-04-26T20:26:03.344678+0000","last_became_active":"2024-04-26T20:25:57.984262+0000","last_became_peered":"2024-04-26T20:25:57.984262+0000","last_unstale":"2024-04-26T20:26:03.344678+0000","last_undegraded":"2024-04-26T20:26:03.344678+0000","last_fullsized":"2024-04-26T20:26:03.344678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:03.002879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345671+0000","last_change":"2024-04-26T20:25:56.989467+0000","last_active":"2024-04-26T20:26:03.345671+0000","last_peered":"2024-04-26T20:26:03.345671+0000","last_clean":"2024-04-26T20:26:03.345671+0000","last_became_active":"2024-04-26T20:25:56.989183+0000","last_became_peered":"2024-04-26T20:25:56.989183+0000","last_unstale":"2024-04-26T20:26:03.345671+0000","last_undegraded":"2024-04-26T20:26:03.345671+0000","last_fullsized":"2024-04-26T20:26:03.345671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:11.650840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346929+0000","last_change":"2024-04-26T20:26:02.414455+0000","last_active":"2024-04-26T20:26:03.346929+0000","last_peered":"2024-04-26T20:26:03.346929+0000","last_clean":"2024-04-26T20:26:03.346929+0000","last_became_active":"2024-04-26T20:26:02.414064+0000","last_became_peered":"2024-04-26T20:26:02.414064+0000","last_unstale":"2024-04-26T20:26:03.346929+0000","last_undegraded":"2024-04-26T20:26:03.346929+0000","last_fullsized":"2024-04-26T20:26:03.346929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093496+0000","last_change":"2024-04-26T20:25:58.008847+0000","last_active":"2024-04-26T20:26:03.093496+0000","last_peered":"2024-04-26T20:26:03.093496+0000","last_clean":"2024-04-26T20:26:03.093496+0000","last_became_active":"2024-04-26T20:25:58.008549+0000","last_became_peered":"2024-04-26T20:25:58.008549+0000","last_unstale":"2024-04-26T20:26:03.093496+0000","last_undegraded":"2024-04-26T20:26:03.093496+0000","last_fullsized":"2024-04-26T20:26:03.093496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345379+0000","last_change":"2024-04-26T20:25:56.981771+0000","last_active":"2024-04-26T20:26:03.345379+0000","last_peered":"2024-04-26T20:26:03.345379+0000","last_clean":"2024-04-26T20:26:03.345379+0000","last_became_active":"2024-04-26T20:25:56.981406+0000","last_became_peered":"2024-04-26T20:25:56.981406+0000","last_unstale":"2024-04-26T20:26:03.345379+0000","last_undegraded":"2024-04-26T20:26:03.345379+0000","last_fullsized":"2024-04-26T20:26:03.345379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345981+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:26:03.345981+0000","last_peered":"2024-04-26T20:26:03.345981+0000","last_clean":"2024-04-26T20:26:03.345981+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:26:03.345981+0000","last_undegraded":"2024-04-26T20:26:03.345981+0000","last_fullsized":"2024-04-26T20:26:03.345981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351315+0000","last_change":"2024-04-26T20:26:02.407562+0000","last_active":"2024-04-26T20:26:03.351315+0000","last_peered":"2024-04-26T20:26:03.351315+0000","last_clean":"2024-04-26T20:26:03.351315+0000","last_became_active":"2024-04-26T20:26:02.406288+0000","last_became_peered":"2024-04-26T20:26:02.406288+0000","last_unstale":"2024-04-26T20:26:03.351315+0000","last_undegraded":"2024-04-26T20:26:03.351315+0000","last_fullsized":"2024-04-26T20:26:03.351315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347732+0000","last_change":"2024-04-26T20:25:58.014416+0000","last_active":"2024-04-26T20:26:03.347732+0000","last_peered":"2024-04-26T20:26:03.347732+0000","last_clean":"2024-04-26T20:26:03.347732+0000","last_became_active":"2024-04-26T20:25:58.014128+0000","last_became_peered":"2024-04-26T20:25:58.014128+0000","last_unstale":"2024-04-26T20:26:03.347732+0000","last_undegraded":"2024-04-26T20:26:03.347732+0000","last_fullsized":"2024-04-26T20:26:03.347732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:22.089301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048942+0000","last_change":"2024-04-26T20:25:56.974272+0000","last_active":"2024-04-26T20:26:03.048942+0000","last_peered":"2024-04-26T20:26:03.048942+0000","last_clean":"2024-04-26T20:26:03.048942+0000","last_became_active":"2024-04-26T20:25:56.972999+0000","last_became_peered":"2024-04-26T20:25:56.972999+0000","last_unstale":"2024-04-26T20:26:03.048942+0000","last_undegraded":"2024-04-26T20:26:03.048942+0000","last_fullsized":"2024-04-26T20:26:03.048942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349430+0000","last_change":"2024-04-26T20:26:02.402839+0000","last_active":"2024-04-26T20:26:03.349430+0000","last_peered":"2024-04-26T20:26:03.349430+0000","last_clean":"2024-04-26T20:26:03.349430+0000","last_became_active":"2024-04-26T20:26:02.401630+0000","last_became_peered":"2024-04-26T20:26:02.401630+0000","last_unstale":"2024-04-26T20:26:03.349430+0000","last_undegraded":"2024-04-26T20:26:03.349430+0000","last_fullsized":"2024-04-26T20:26:03.349430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346830+0000","last_change":"2024-04-26T20:25:58.009012+0000","last_active":"2024-04-26T20:26:03.346830+0000","last_peered":"2024-04-26T20:26:03.346830+0000","last_clean":"2024-04-26T20:26:03.346830+0000","last_became_active":"2024-04-26T20:25:58.008594+0000","last_became_peered":"2024-04-26T20:25:58.008594+0000","last_unstale":"2024-04-26T20:26:03.346830+0000","last_undegraded":"2024-04-26T20:26:03.346830+0000","last_fullsized":"2024-04-26T20:26:03.346830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344764+0000","last_change":"2024-04-26T20:25:56.972043+0000","last_active":"2024-04-26T20:26:03.344764+0000","last_peered":"2024-04-26T20:26:03.344764+0000","last_clean":"2024-04-26T20:26:03.344764+0000","last_became_active":"2024-04-26T20:25:56.971671+0000","last_became_peered":"2024-04-26T20:25:56.971671+0000","last_unstale":"2024-04-26T20:26:03.344764+0000","last_undegraded":"2024-04-26T20:26:03.344764+0000","last_fullsized":"2024-04-26T20:26:03.344764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346513+0000","last_change":"2024-04-26T20:26:02.073728+0000","last_active":"2024-04-26T20:26:03.346513+0000","last_peered":"2024-04-26T20:26:03.346513+0000","last_clean":"2024-04-26T20:26:03.346513+0000","last_became_active":"2024-04-26T20:26:02.073238+0000","last_became_peered":"2024-04-26T20:26:02.073238+0000","last_unstale":"2024-04-26T20:26:03.346513+0000","last_undegraded":"2024-04-26T20:26:03.346513+0000","last_fullsized":"2024-04-26T20:26:03.346513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350204+0000","last_change":"2024-04-26T20:25:58.011539+0000","last_active":"2024-04-26T20:26:03.350204+0000","last_peered":"2024-04-26T20:26:03.350204+0000","last_clean":"2024-04-26T20:26:03.350204+0000","last_became_active":"2024-04-26T20:25:58.011256+0000","last_became_peered":"2024-04-26T20:25:58.011256+0000","last_unstale":"2024-04-26T20:26:03.350204+0000","last_undegraded":"2024-04-26T20:26:03.350204+0000","last_fullsized":"2024-04-26T20:26:03.350204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:02.292377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048755+0000","last_change":"2024-04-26T20:25:56.972332+0000","last_active":"2024-04-26T20:26:03.048755+0000","last_peered":"2024-04-26T20:26:03.048755+0000","last_clean":"2024-04-26T20:26:03.048755+0000","last_became_active":"2024-04-26T20:25:56.972075+0000","last_became_peered":"2024-04-26T20:25:56.972075+0000","last_unstale":"2024-04-26T20:26:03.048755+0000","last_undegraded":"2024-04-26T20:26:03.048755+0000","last_fullsized":"2024-04-26T20:26:03.048755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346282+0000","last_change":"2024-04-26T20:26:02.414970+0000","last_active":"2024-04-26T20:26:03.346282+0000","last_peered":"2024-04-26T20:26:03.346282+0000","last_clean":"2024-04-26T20:26:03.346282+0000","last_became_active":"2024-04-26T20:26:02.414093+0000","last_became_peered":"2024-04-26T20:26:02.414093+0000","last_unstale":"2024-04-26T20:26:03.346282+0000","last_undegraded":"2024-04-26T20:26:03.346282+0000","last_fullsized":"2024-04-26T20:26:03.346282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348952+0000","last_change":"2024-04-26T20:25:58.014604+0000","last_active":"2024-04-26T20:26:03.348952+0000","last_peered":"2024-04-26T20:26:03.348952+0000","last_clean":"2024-04-26T20:26:03.348952+0000","last_became_active":"2024-04-26T20:25:58.014239+0000","last_became_peered":"2024-04-26T20:25:58.014239+0000","last_unstale":"2024-04-26T20:26:03.348952+0000","last_undegraded":"2024-04-26T20:26:03.348952+0000","last_fullsized":"2024-04-26T20:26:03.348952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:24.555699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050237+0000","last_change":"2024-04-26T20:25:56.970496+0000","last_active":"2024-04-26T20:26:03.050237+0000","last_peered":"2024-04-26T20:26:03.050237+0000","last_clean":"2024-04-26T20:26:03.050237+0000","last_became_active":"2024-04-26T20:25:56.970174+0000","last_became_peered":"2024-04-26T20:25:56.970174+0000","last_unstale":"2024-04-26T20:26:03.050237+0000","last_undegraded":"2024-04-26T20:26:03.050237+0000","last_fullsized":"2024-04-26T20:26:03.050237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:25.599135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049091+0000","last_change":"2024-04-26T20:26:02.082060+0000","last_active":"2024-04-26T20:26:03.049091+0000","last_peered":"2024-04-26T20:26:03.049091+0000","last_clean":"2024-04-26T20:26:03.049091+0000","last_became_active":"2024-04-26T20:26:02.081533+0000","last_became_peered":"2024-04-26T20:26:02.081533+0000","last_unstale":"2024-04-26T20:26:03.049091+0000","last_undegraded":"2024-04-26T20:26:03.049091+0000","last_fullsized":"2024-04-26T20:26:03.049091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:02.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346324+0000","last_change":"2024-04-26T20:25:58.008801+0000","last_active":"2024-04-26T20:26:03.346324+0000","last_peered":"2024-04-26T20:26:03.346324+0000","last_clean":"2024-04-26T20:26:03.346324+0000","last_became_active":"2024-04-26T20:25:58.008479+0000","last_became_peered":"2024-04-26T20:25:58.008479+0000","last_unstale":"2024-04-26T20:26:03.346324+0000","last_undegraded":"2024-04-26T20:26:03.346324+0000","last_fullsized":"2024-04-26T20:26:03.346324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047755+0000","last_change":"2024-04-26T20:25:56.968431+0000","last_active":"2024-04-26T20:26:03.047755+0000","last_peered":"2024-04-26T20:26:03.047755+0000","last_clean":"2024-04-26T20:26:03.047755+0000","last_became_active":"2024-04-26T20:25:56.967495+0000","last_became_peered":"2024-04-26T20:25:56.967495+0000","last_unstale":"2024-04-26T20:26:03.047755+0000","last_undegraded":"2024-04-26T20:26:03.047755+0000","last_fullsized":"2024-04-26T20:26:03.047755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346734+0000","last_change":"2024-04-26T20:26:02.078801+0000","last_active":"2024-04-26T20:26:03.346734+0000","last_peered":"2024-04-26T20:26:03.346734+0000","last_clean":"2024-04-26T20:26:03.346734+0000","last_became_active":"2024-04-26T20:26:02.078343+0000","last_became_peered":"2024-04-26T20:26:02.078343+0000","last_unstale":"2024-04-26T20:26:03.346734+0000","last_undegraded":"2024-04-26T20:26:03.346734+0000","last_fullsized":"2024-04-26T20:26:03.346734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350880+0000","last_change":"2024-04-26T20:25:57.996180+0000","last_active":"2024-04-26T20:26:03.350880+0000","last_peered":"2024-04-26T20:26:03.350880+0000","last_clean":"2024-04-26T20:26:03.350880+0000","last_became_active":"2024-04-26T20:25:57.990792+0000","last_became_peered":"2024-04-26T20:25:57.990792+0000","last_unstale":"2024-04-26T20:26:03.350880+0000","last_undegraded":"2024-04-26T20:26:03.350880+0000","last_fullsized":"2024-04-26T20:26:03.350880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349887+0000","last_change":"2024-04-26T20:25:56.974115+0000","last_active":"2024-04-26T20:26:03.349887+0000","last_peered":"2024-04-26T20:26:03.349887+0000","last_clean":"2024-04-26T20:26:03.349887+0000","last_became_active":"2024-04-26T20:25:56.973801+0000","last_became_peered":"2024-04-26T20:25:56.973801+0000","last_unstale":"2024-04-26T20:26:03.349887+0000","last_undegraded":"2024-04-26T20:26:03.349887+0000","last_fullsized":"2024-04-26T20:26:03.349887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095178+0000","last_change":"2024-04-26T20:26:02.413590+0000","last_active":"2024-04-26T20:26:03.095178+0000","last_peered":"2024-04-26T20:26:03.095178+0000","last_clean":"2024-04-26T20:26:03.095178+0000","last_became_active":"2024-04-26T20:26:02.413104+0000","last_became_peered":"2024-04-26T20:26:02.413104+0000","last_unstale":"2024-04-26T20:26:03.095178+0000","last_undegraded":"2024-04-26T20:26:03.095178+0000","last_fullsized":"2024-04-26T20:26:03.095178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:35.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347726+0000","last_change":"2024-04-26T20:25:58.014095+0000","last_active":"2024-04-26T20:26:03.347726+0000","last_peered":"2024-04-26T20:26:03.347726+0000","last_clean":"2024-04-26T20:26:03.347726+0000","last_became_active":"2024-04-26T20:25:58.012813+0000","last_became_peered":"2024-04-26T20:25:58.012813+0000","last_unstale":"2024-04-26T20:26:03.347726+0000","last_undegraded":"2024-04-26T20:26:03.347726+0000","last_fullsized":"2024-04-26T20:26:03.347726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348384+0000","last_change":"2024-04-26T20:25:56.987103+0000","last_active":"2024-04-26T20:26:03.348384+0000","last_peered":"2024-04-26T20:26:03.348384+0000","last_clean":"2024-04-26T20:26:03.348384+0000","last_became_active":"2024-04-26T20:25:56.986766+0000","last_became_peered":"2024-04-26T20:25:56.986766+0000","last_unstale":"2024-04-26T20:26:03.348384+0000","last_undegraded":"2024-04-26T20:26:03.348384+0000","last_fullsized":"2024-04-26T20:26:03.348384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094102+0000","last_change":"2024-04-26T20:26:02.400999+0000","last_active":"2024-04-26T20:26:03.094102+0000","last_peered":"2024-04-26T20:26:03.094102+0000","last_clean":"2024-04-26T20:26:03.094102+0000","last_became_active":"2024-04-26T20:26:02.399994+0000","last_became_peered":"2024-04-26T20:26:02.399994+0000","last_unstale":"2024-04-26T20:26:03.094102+0000","last_undegraded":"2024-04-26T20:26:03.094102+0000","last_fullsized":"2024-04-26T20:26:03.094102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:05.311868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345529+0000","last_change":"2024-04-26T20:25:57.996909+0000","last_active":"2024-04-26T20:26:03.345529+0000","last_peered":"2024-04-26T20:26:03.345529+0000","last_clean":"2024-04-26T20:26:03.345529+0000","last_became_active":"2024-04-26T20:25:57.996635+0000","last_became_peered":"2024-04-26T20:25:57.996635+0000","last_unstale":"2024-04-26T20:26:03.345529+0000","last_undegraded":"2024-04-26T20:26:03.345529+0000","last_fullsized":"2024-04-26T20:26:03.345529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346908+0000","last_change":"2024-04-26T20:25:56.974019+0000","last_active":"2024-04-26T20:26:03.346908+0000","last_peered":"2024-04-26T20:26:03.346908+0000","last_clean":"2024-04-26T20:26:03.346908+0000","last_became_active":"2024-04-26T20:25:56.971170+0000","last_became_peered":"2024-04-26T20:25:56.971170+0000","last_unstale":"2024-04-26T20:26:03.346908+0000","last_undegraded":"2024-04-26T20:26:03.346908+0000","last_fullsized":"2024-04-26T20:26:03.346908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347253+0000","last_change":"2024-04-26T20:26:02.402012+0000","last_active":"2024-04-26T20:26:03.347253+0000","last_peered":"2024-04-26T20:26:03.347253+0000","last_clean":"2024-04-26T20:26:03.347253+0000","last_became_active":"2024-04-26T20:26:02.400605+0000","last_became_peered":"2024-04-26T20:26:02.400605+0000","last_unstale":"2024-04-26T20:26:03.347253+0000","last_undegraded":"2024-04-26T20:26:03.347253+0000","last_fullsized":"2024-04-26T20:26:03.347253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347168+0000","last_change":"2024-04-26T20:25:58.001764+0000","last_active":"2024-04-26T20:26:03.347168+0000","last_peered":"2024-04-26T20:26:03.347168+0000","last_clean":"2024-04-26T20:26:03.347168+0000","last_became_active":"2024-04-26T20:25:58.001502+0000","last_became_peered":"2024-04-26T20:25:58.001502+0000","last_unstale":"2024-04-26T20:26:03.347168+0000","last_undegraded":"2024-04-26T20:26:03.347168+0000","last_fullsized":"2024-04-26T20:26:03.347168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:16.030394+0000","last_change":"2024-04-26T20:25:56.984259+0000","last_active":"2024-04-26T20:26:16.030394+0000","last_peered":"2024-04-26T20:26:16.030394+0000","last_clean":"2024-04-26T20:26:16.030394+0000","last_became_active":"2024-04-26T20:25:56.983974+0000","last_became_peered":"2024-04-26T20:25:56.983974+0000","last_unstale":"2024-04-26T20:26:16.030394+0000","last_undegraded":"2024-04-26T20:26:16.030394+0000","last_fullsized":"2024-04-26T20:26:16.030394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345142+0000","last_change":"2024-04-26T20:26:02.080802+0000","last_active":"2024-04-26T20:26:03.345142+0000","last_peered":"2024-04-26T20:26:03.345142+0000","last_clean":"2024-04-26T20:26:03.345142+0000","last_became_active":"2024-04-26T20:26:02.079790+0000","last_became_peered":"2024-04-26T20:26:02.079790+0000","last_unstale":"2024-04-26T20:26:03.345142+0000","last_undegraded":"2024-04-26T20:26:03.345142+0000","last_fullsized":"2024-04-26T20:26:03.345142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050516+0000","last_change":"2024-04-26T20:25:57.999695+0000","last_active":"2024-04-26T20:26:03.050516+0000","last_peered":"2024-04-26T20:26:03.050516+0000","last_clean":"2024-04-26T20:26:03.050516+0000","last_became_active":"2024-04-26T20:25:57.999476+0000","last_became_peered":"2024-04-26T20:25:57.999476+0000","last_unstale":"2024-04-26T20:26:03.050516+0000","last_undegraded":"2024-04-26T20:26:03.050516+0000","last_fullsized":"2024-04-26T20:26:03.050516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344567+0000","last_change":"2024-04-26T20:25:56.984463+0000","last_active":"2024-04-26T20:26:03.344567+0000","last_peered":"2024-04-26T20:26:03.344567+0000","last_clean":"2024-04-26T20:26:03.344567+0000","last_became_active":"2024-04-26T20:25:56.984176+0000","last_became_peered":"2024-04-26T20:25:56.984176+0000","last_unstale":"2024-04-26T20:26:03.344567+0000","last_undegraded":"2024-04-26T20:26:03.344567+0000","last_fullsized":"2024-04-26T20:26:03.344567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051617+0000","last_change":"2024-04-26T20:26:02.079338+0000","last_active":"2024-04-26T20:26:03.051617+0000","last_peered":"2024-04-26T20:26:03.051617+0000","last_clean":"2024-04-26T20:26:03.051617+0000","last_became_active":"2024-04-26T20:26:02.078830+0000","last_became_peered":"2024-04-26T20:26:02.078830+0000","last_unstale":"2024-04-26T20:26:03.051617+0000","last_undegraded":"2024-04-26T20:26:03.051617+0000","last_fullsized":"2024-04-26T20:26:03.051617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347553+0000","last_change":"2024-04-26T20:25:58.008157+0000","last_active":"2024-04-26T20:26:03.347553+0000","last_peered":"2024-04-26T20:26:03.347553+0000","last_clean":"2024-04-26T20:26:03.347553+0000","last_became_active":"2024-04-26T20:25:58.005542+0000","last_became_peered":"2024-04-26T20:25:58.005542+0000","last_unstale":"2024-04-26T20:26:03.347553+0000","last_undegraded":"2024-04-26T20:26:03.347553+0000","last_fullsized":"2024-04-26T20:26:03.347553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348327+0000","last_change":"2024-04-26T20:25:56.987547+0000","last_active":"2024-04-26T20:26:03.348327+0000","last_peered":"2024-04-26T20:26:03.348327+0000","last_clean":"2024-04-26T20:26:03.348327+0000","last_became_active":"2024-04-26T20:25:56.985375+0000","last_became_peered":"2024-04-26T20:25:56.985375+0000","last_unstale":"2024-04-26T20:26:03.348327+0000","last_undegraded":"2024-04-26T20:26:03.348327+0000","last_fullsized":"2024-04-26T20:26:03.348327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093045+0000","last_change":"2024-04-26T20:26:02.073521+0000","last_active":"2024-04-26T20:26:03.093045+0000","last_peered":"2024-04-26T20:26:03.093045+0000","last_clean":"2024-04-26T20:26:03.093045+0000","last_became_active":"2024-04-26T20:26:02.073134+0000","last_became_peered":"2024-04-26T20:26:02.073134+0000","last_unstale":"2024-04-26T20:26:03.093045+0000","last_undegraded":"2024-04-26T20:26:03.093045+0000","last_fullsized":"2024-04-26T20:26:03.093045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:06.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348491+0000","last_change":"2024-04-26T20:25:58.011178+0000","last_active":"2024-04-26T20:26:03.348491+0000","last_peered":"2024-04-26T20:26:03.348491+0000","last_clean":"2024-04-26T20:26:03.348491+0000","last_became_active":"2024-04-26T20:25:58.010833+0000","last_became_peered":"2024-04-26T20:25:58.010833+0000","last_unstale":"2024-04-26T20:26:03.348491+0000","last_undegraded":"2024-04-26T20:26:03.348491+0000","last_fullsized":"2024-04-26T20:26:03.348491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351644+0000","last_change":"2024-04-26T20:25:56.986174+0000","last_active":"2024-04-26T20:26:03.351644+0000","last_peered":"2024-04-26T20:26:03.351644+0000","last_clean":"2024-04-26T20:26:03.351644+0000","last_became_active":"2024-04-26T20:25:56.985869+0000","last_became_peered":"2024-04-26T20:25:56.985869+0000","last_unstale":"2024-04-26T20:26:03.351644+0000","last_undegraded":"2024-04-26T20:26:03.351644+0000","last_fullsized":"2024-04-26T20:26:03.351644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093097+0000","last_change":"2024-04-26T20:26:02.413559+0000","last_active":"2024-04-26T20:26:03.093097+0000","last_peered":"2024-04-26T20:26:03.093097+0000","last_clean":"2024-04-26T20:26:03.093097+0000","last_became_active":"2024-04-26T20:26:02.413096+0000","last_became_peered":"2024-04-26T20:26:02.413096+0000","last_unstale":"2024-04-26T20:26:03.093097+0000","last_undegraded":"2024-04-26T20:26:03.093097+0000","last_fullsized":"2024-04-26T20:26:03.093097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:44.245008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346456+0000","last_change":"2024-04-26T20:26:02.413921+0000","last_active":"2024-04-26T20:26:03.346456+0000","last_peered":"2024-04-26T20:26:03.346456+0000","last_clean":"2024-04-26T20:26:03.346456+0000","last_became_active":"2024-04-26T20:26:02.413388+0000","last_became_peered":"2024-04-26T20:26:02.413388+0000","last_unstale":"2024-04-26T20:26:03.346456+0000","last_undegraded":"2024-04-26T20:26:03.346456+0000","last_fullsized":"2024-04-26T20:26:03.346456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345276+0000","last_change":"2024-04-26T20:25:58.004583+0000","last_active":"2024-04-26T20:26:03.345276+0000","last_peered":"2024-04-26T20:26:03.345276+0000","last_clean":"2024-04-26T20:26:03.345276+0000","last_became_active":"2024-04-26T20:25:58.004228+0000","last_became_peered":"2024-04-26T20:25:58.004228+0000","last_unstale":"2024-04-26T20:26:03.345276+0000","last_undegraded":"2024-04-26T20:26:03.345276+0000","last_fullsized":"2024-04-26T20:26:03.345276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346675+0000","last_change":"2024-04-26T20:25:56.985910+0000","last_active":"2024-04-26T20:26:03.346675+0000","last_peered":"2024-04-26T20:26:03.346675+0000","last_clean":"2024-04-26T20:26:03.346675+0000","last_became_active":"2024-04-26T20:25:56.985636+0000","last_became_peered":"2024-04-26T20:25:56.985636+0000","last_unstale":"2024-04-26T20:26:03.346675+0000","last_undegraded":"2024-04-26T20:26:03.346675+0000","last_fullsized":"2024-04-26T20:26:03.346675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351558+0000","last_change":"2024-04-26T20:26:02.079470+0000","last_active":"2024-04-26T20:26:03.351558+0000","last_peered":"2024-04-26T20:26:03.351558+0000","last_clean":"2024-04-26T20:26:03.351558+0000","last_became_active":"2024-04-26T20:26:02.078853+0000","last_became_peered":"2024-04-26T20:26:02.078853+0000","last_unstale":"2024-04-26T20:26:03.351558+0000","last_undegraded":"2024-04-26T20:26:03.351558+0000","last_fullsized":"2024-04-26T20:26:03.351558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093119+0000","last_change":"2024-04-26T20:25:58.006907+0000","last_active":"2024-04-26T20:26:03.093119+0000","last_peered":"2024-04-26T20:26:03.093119+0000","last_clean":"2024-04-26T20:26:03.093119+0000","last_became_active":"2024-04-26T20:25:58.006370+0000","last_became_peered":"2024-04-26T20:25:58.006370+0000","last_unstale":"2024-04-26T20:26:03.093119+0000","last_undegraded":"2024-04-26T20:26:03.093119+0000","last_fullsized":"2024-04-26T20:26:03.093119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093249+0000","last_change":"2024-04-26T20:25:56.982952+0000","last_active":"2024-04-26T20:26:03.093249+0000","last_peered":"2024-04-26T20:26:03.093249+0000","last_clean":"2024-04-26T20:26:03.093249+0000","last_became_active":"2024-04-26T20:25:56.982240+0000","last_became_peered":"2024-04-26T20:25:56.982240+0000","last_unstale":"2024-04-26T20:26:03.093249+0000","last_undegraded":"2024-04-26T20:26:03.093249+0000","last_fullsized":"2024-04-26T20:26:03.093249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349655+0000","last_change":"2024-04-26T20:25:56.982228+0000","last_active":"2024-04-26T20:26:03.349655+0000","last_peered":"2024-04-26T20:26:03.349655+0000","last_clean":"2024-04-26T20:26:03.349655+0000","last_became_active":"2024-04-26T20:25:56.981083+0000","last_became_peered":"2024-04-26T20:25:56.981083+0000","last_unstale":"2024-04-26T20:26:03.349655+0000","last_undegraded":"2024-04-26T20:26:03.349655+0000","last_fullsized":"2024-04-26T20:26:03.349655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346695+0000","last_change":"2024-04-26T20:25:58.001685+0000","last_active":"2024-04-26T20:26:03.346695+0000","last_peered":"2024-04-26T20:26:03.346695+0000","last_clean":"2024-04-26T20:26:03.346695+0000","last_became_active":"2024-04-26T20:25:58.001160+0000","last_became_peered":"2024-04-26T20:25:58.001160+0000","last_unstale":"2024-04-26T20:26:03.346695+0000","last_undegraded":"2024-04-26T20:26:03.346695+0000","last_fullsized":"2024-04-26T20:26:03.346695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048776+0000","last_change":"2024-04-26T20:26:02.078797+0000","last_active":"2024-04-26T20:26:03.048776+0000","last_peered":"2024-04-26T20:26:03.048776+0000","last_clean":"2024-04-26T20:26:03.048776+0000","last_became_active":"2024-04-26T20:26:02.078444+0000","last_became_peered":"2024-04-26T20:26:02.078444+0000","last_unstale":"2024-04-26T20:26:03.048776+0000","last_undegraded":"2024-04-26T20:26:03.048776+0000","last_fullsized":"2024-04-26T20:26:03.048776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:16.872808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093946+0000","last_change":"2024-04-26T20:25:56.986739+0000","last_active":"2024-04-26T20:26:03.093946+0000","last_peered":"2024-04-26T20:26:03.093946+0000","last_clean":"2024-04-26T20:26:03.093946+0000","last_became_active":"2024-04-26T20:25:56.986399+0000","last_became_peered":"2024-04-26T20:25:56.986399+0000","last_unstale":"2024-04-26T20:26:03.093946+0000","last_undegraded":"2024-04-26T20:26:03.093946+0000","last_fullsized":"2024-04-26T20:26:03.093946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050966+0000","last_change":"2024-04-26T20:25:57.999363+0000","last_active":"2024-04-26T20:26:03.050966+0000","last_peered":"2024-04-26T20:26:03.050966+0000","last_clean":"2024-04-26T20:26:03.050966+0000","last_became_active":"2024-04-26T20:25:57.995297+0000","last_became_peered":"2024-04-26T20:25:57.995297+0000","last_unstale":"2024-04-26T20:26:03.050966+0000","last_undegraded":"2024-04-26T20:26:03.050966+0000","last_fullsized":"2024-04-26T20:26:03.050966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.671294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094404+0000","last_change":"2024-04-26T20:26:02.075522+0000","last_active":"2024-04-26T20:26:03.094404+0000","last_peered":"2024-04-26T20:26:03.094404+0000","last_clean":"2024-04-26T20:26:03.094404+0000","last_became_active":"2024-04-26T20:26:02.075089+0000","last_became_peered":"2024-04-26T20:26:02.075089+0000","last_unstale":"2024-04-26T20:26:03.094404+0000","last_undegraded":"2024-04-26T20:26:03.094404+0000","last_fullsized":"2024-04-26T20:26:03.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:23.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350468+0000","last_change":"2024-04-26T20:25:57.990722+0000","last_active":"2024-04-26T20:26:03.350468+0000","last_peered":"2024-04-26T20:26:03.350468+0000","last_clean":"2024-04-26T20:26:03.350468+0000","last_became_active":"2024-04-26T20:25:57.990382+0000","last_became_peered":"2024-04-26T20:25:57.990382+0000","last_unstale":"2024-04-26T20:26:03.350468+0000","last_undegraded":"2024-04-26T20:26:03.350468+0000","last_fullsized":"2024-04-26T20:26:03.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047754+0000","last_change":"2024-04-26T20:25:58.014226+0000","last_active":"2024-04-26T20:26:03.047754+0000","last_peered":"2024-04-26T20:26:03.047754+0000","last_clean":"2024-04-26T20:26:03.047754+0000","last_became_active":"2024-04-26T20:25:58.013319+0000","last_became_peered":"2024-04-26T20:25:58.013319+0000","last_unstale":"2024-04-26T20:26:03.047754+0000","last_undegraded":"2024-04-26T20:26:03.047754+0000","last_fullsized":"2024-04-26T20:26:03.047754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348155+0000","last_change":"2024-04-26T20:25:58.008173+0000","last_active":"2024-04-26T20:26:03.348155+0000","last_peered":"2024-04-26T20:26:03.348155+0000","last_clean":"2024-04-26T20:26:03.348155+0000","last_became_active":"2024-04-26T20:25:58.007377+0000","last_became_peered":"2024-04-26T20:25:58.007377+0000","last_unstale":"2024-04-26T20:26:03.348155+0000","last_undegraded":"2024-04-26T20:26:03.348155+0000","last_fullsized":"2024-04-26T20:26:03.348155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346409+0000","last_change":"2024-04-26T20:25:58.003184+0000","last_active":"2024-04-26T20:26:03.346409+0000","last_peered":"2024-04-26T20:26:03.346409+0000","last_clean":"2024-04-26T20:26:03.346409+0000","last_became_active":"2024-04-26T20:25:58.002849+0000","last_became_peered":"2024-04-26T20:25:58.002849+0000","last_unstale":"2024-04-26T20:26:03.346409+0000","last_undegraded":"2024-04-26T20:26:03.346409+0000","last_fullsized":"2024-04-26T20:26:03.346409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094328+0000","last_change":"2024-04-26T20:25:57.994419+0000","last_active":"2024-04-26T20:26:03.094328+0000","last_peered":"2024-04-26T20:26:03.094328+0000","last_clean":"2024-04-26T20:26:03.094328+0000","last_became_active":"2024-04-26T20:25:57.994077+0000","last_became_peered":"2024-04-26T20:25:57.994077+0000","last_unstale":"2024-04-26T20:26:03.094328+0000","last_undegraded":"2024-04-26T20:26:03.094328+0000","last_fullsized":"2024-04-26T20:26:03.094328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344289+0000","last_change":"2024-04-26T20:25:58.000446+0000","last_active":"2024-04-26T20:26:03.344289+0000","last_peered":"2024-04-26T20:26:03.344289+0000","last_clean":"2024-04-26T20:26:03.344289+0000","last_became_active":"2024-04-26T20:25:58.000166+0000","last_became_peered":"2024-04-26T20:25:58.000166+0000","last_unstale":"2024-04-26T20:26:03.344289+0000","last_undegraded":"2024-04-26T20:26:03.344289+0000","last_fullsized":"2024-04-26T20:26:03.344289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351690+0000","last_change":"2024-04-26T20:25:57.996288+0000","last_active":"2024-04-26T20:26:03.351690+0000","last_peered":"2024-04-26T20:26:03.351690+0000","last_clean":"2024-04-26T20:26:03.351690+0000","last_became_active":"2024-04-26T20:25:57.995978+0000","last_became_peered":"2024-04-26T20:25:57.995978+0000","last_unstale":"2024-04-26T20:26:03.351690+0000","last_undegraded":"2024-04-26T20:26:03.351690+0000","last_fullsized":"2024-04-26T20:26:03.351690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094950+0000","last_change":"2024-04-26T20:25:57.993394+0000","last_active":"2024-04-26T20:26:03.094950+0000","last_peered":"2024-04-26T20:26:03.094950+0000","last_clean":"2024-04-26T20:26:03.094950+0000","last_became_active":"2024-04-26T20:25:57.993023+0000","last_became_peered":"2024-04-26T20:25:57.993023+0000","last_unstale":"2024-04-26T20:26:03.094950+0000","last_undegraded":"2024-04-26T20:26:03.094950+0000","last_fullsized":"2024-04-26T20:26:03.094950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346415+0000","last_change":"2024-04-26T20:25:57.997685+0000","last_active":"2024-04-26T20:26:03.346415+0000","last_peered":"2024-04-26T20:26:03.346415+0000","last_clean":"2024-04-26T20:26:03.346415+0000","last_became_active":"2024-04-26T20:25:57.997422+0000","last_became_peered":"2024-04-26T20:25:57.997422+0000","last_unstale":"2024-04-26T20:26:03.346415+0000","last_undegraded":"2024-04-26T20:26:03.346415+0000","last_fullsized":"2024-04-26T20:26:03.346415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093095+0000","last_change":"2024-04-26T20:25:58.003864+0000","last_active":"2024-04-26T20:26:03.093095+0000","last_peered":"2024-04-26T20:26:03.093095+0000","last_clean":"2024-04-26T20:26:03.093095+0000","last_became_active":"2024-04-26T20:25:58.003553+0000","last_became_peered":"2024-04-26T20:25:58.003553+0000","last_unstale":"2024-04-26T20:26:03.093095+0000","last_undegraded":"2024-04-26T20:26:03.093095+0000","last_fullsized":"2024-04-26T20:26:03.093095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347138+0000","last_change":"2024-04-26T20:25:58.007689+0000","last_active":"2024-04-26T20:26:03.347138+0000","last_peered":"2024-04-26T20:26:03.347138+0000","last_clean":"2024-04-26T20:26:03.347138+0000","last_became_active":"2024-04-26T20:25:58.006743+0000","last_became_peered":"2024-04-26T20:25:58.006743+0000","last_unstale":"2024-04-26T20:26:03.347138+0000","last_undegraded":"2024-04-26T20:26:03.347138+0000","last_fullsized":"2024-04-26T20:26:03.347138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346030+0000","last_change":"2024-04-26T20:25:58.001516+0000","last_active":"2024-04-26T20:26:03.346030+0000","last_peered":"2024-04-26T20:26:03.346030+0000","last_clean":"2024-04-26T20:26:03.346030+0000","last_became_active":"2024-04-26T20:25:58.001241+0000","last_became_peered":"2024-04-26T20:25:58.001241+0000","last_unstale":"2024-04-26T20:26:03.346030+0000","last_undegraded":"2024-04-26T20:26:03.346030+0000","last_fullsized":"2024-04-26T20:26:03.346030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350860+0000","last_change":"2024-04-26T20:25:57.993316+0000","last_active":"2024-04-26T20:26:03.350860+0000","last_peered":"2024-04-26T20:26:03.350860+0000","last_clean":"2024-04-26T20:26:03.350860+0000","last_became_active":"2024-04-26T20:25:57.991497+0000","last_became_peered":"2024-04-26T20:25:57.991497+0000","last_unstale":"2024-04-26T20:26:03.350860+0000","last_undegraded":"2024-04-26T20:26:03.350860+0000","last_fullsized":"2024-04-26T20:26:03.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348333+0000","last_change":"2024-04-26T20:25:58.004031+0000","last_active":"2024-04-26T20:26:03.348333+0000","last_peered":"2024-04-26T20:26:03.348333+0000","last_clean":"2024-04-26T20:26:03.348333+0000","last_became_active":"2024-04-26T20:25:58.003728+0000","last_became_peered":"2024-04-26T20:25:58.003728+0000","last_unstale":"2024-04-26T20:26:03.348333+0000","last_undegraded":"2024-04-26T20:26:03.348333+0000","last_fullsized":"2024-04-26T20:26:03.348333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:28.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048584+0000","last_change":"2024-04-26T20:25:57.986712+0000","last_active":"2024-04-26T20:26:03.048584+0000","last_peered":"2024-04-26T20:26:03.048584+0000","last_clean":"2024-04-26T20:26:03.048584+0000","last_became_active":"2024-04-26T20:25:57.986233+0000","last_became_peered":"2024-04-26T20:25:57.986233+0000","last_unstale":"2024-04-26T20:26:03.048584+0000","last_undegraded":"2024-04-26T20:26:03.048584+0000","last_fullsized":"2024-04-26T20:26:03.048584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346524+0000","last_change":"2024-04-26T20:25:58.011317+0000","last_active":"2024-04-26T20:26:03.346524+0000","last_peered":"2024-04-26T20:26:03.346524+0000","last_clean":"2024-04-26T20:26:03.346524+0000","last_became_active":"2024-04-26T20:25:58.010881+0000","last_became_peered":"2024-04-26T20:25:58.010881+0000","last_unstale":"2024-04-26T20:26:03.346524+0000","last_undegraded":"2024-04-26T20:26:03.346524+0000","last_fullsized":"2024-04-26T20:26:03.346524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344914+0000","last_change":"2024-04-26T20:25:58.018419+0000","last_active":"2024-04-26T20:26:03.344914+0000","last_peered":"2024-04-26T20:26:03.344914+0000","last_clean":"2024-04-26T20:26:03.344914+0000","last_became_active":"2024-04-26T20:25:58.018134+0000","last_became_peered":"2024-04-26T20:25:58.018134+0000","last_unstale":"2024-04-26T20:26:03.344914+0000","last_undegraded":"2024-04-26T20:26:03.344914+0000","last_fullsized":"2024-04-26T20:26:03.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343902+0000","last_change":"2024-04-26T20:25:58.008263+0000","last_active":"2024-04-26T20:26:03.343902+0000","last_peered":"2024-04-26T20:26:03.343902+0000","last_clean":"2024-04-26T20:26:03.343902+0000","last_became_active":"2024-04-26T20:25:58.007268+0000","last_became_peered":"2024-04-26T20:25:58.007268+0000","last_unstale":"2024-04-26T20:26:03.343902+0000","last_undegraded":"2024-04-26T20:26:03.343902+0000","last_fullsized":"2024-04-26T20:26:03.343902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345391+0000","last_change":"2024-04-26T20:25:58.011832+0000","last_active":"2024-04-26T20:26:03.345391+0000","last_peered":"2024-04-26T20:26:03.345391+0000","last_clean":"2024-04-26T20:26:03.345391+0000","last_became_active":"2024-04-26T20:25:58.011007+0000","last_became_peered":"2024-04-26T20:25:58.011007+0000","last_unstale":"2024-04-26T20:26:03.345391+0000","last_undegraded":"2024-04-26T20:26:03.345391+0000","last_fullsized":"2024-04-26T20:26:03.345391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:38.909585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093783+0000","last_change":"2024-04-26T20:25:57.993780+0000","last_active":"2024-04-26T20:26:03.093783+0000","last_peered":"2024-04-26T20:26:03.093783+0000","last_clean":"2024-04-26T20:26:03.093783+0000","last_became_active":"2024-04-26T20:25:57.993444+0000","last_became_peered":"2024-04-26T20:25:57.993444+0000","last_unstale":"2024-04-26T20:26:03.093783+0000","last_undegraded":"2024-04-26T20:26:03.093783+0000","last_fullsized":"2024-04-26T20:26:03.093783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050252+0000","last_change":"2024-04-26T20:25:57.985098+0000","last_active":"2024-04-26T20:26:03.050252+0000","last_peered":"2024-04-26T20:26:03.050252+0000","last_clean":"2024-04-26T20:26:03.050252+0000","last_became_active":"2024-04-26T20:25:57.984739+0000","last_became_peered":"2024-04-26T20:25:57.984739+0000","last_unstale":"2024-04-26T20:26:03.050252+0000","last_undegraded":"2024-04-26T20:26:03.050252+0000","last_fullsized":"2024-04-26T20:26:03.050252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351223+0000","last_change":"2024-04-26T20:25:58.007442+0000","last_active":"2024-04-26T20:26:03.351223+0000","last_peered":"2024-04-26T20:26:03.351223+0000","last_clean":"2024-04-26T20:26:03.351223+0000","last_became_active":"2024-04-26T20:25:58.007150+0000","last_became_peered":"2024-04-26T20:25:58.007150+0000","last_unstale":"2024-04-26T20:26:03.351223+0000","last_undegraded":"2024-04-26T20:26:03.351223+0000","last_fullsized":"2024-04-26T20:26:03.351223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351686+0000","last_change":"2024-04-26T20:25:58.007696+0000","last_active":"2024-04-26T20:26:03.351686+0000","last_peered":"2024-04-26T20:26:03.351686+0000","last_clean":"2024-04-26T20:26:03.351686+0000","last_became_active":"2024-04-26T20:25:58.007415+0000","last_became_peered":"2024-04-26T20:25:58.007415+0000","last_unstale":"2024-04-26T20:26:03.351686+0000","last_undegraded":"2024-04-26T20:26:03.351686+0000","last_fullsized":"2024-04-26T20:26:03.351686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350018+0000","last_change":"2024-04-26T20:25:58.014108+0000","last_active":"2024-04-26T20:26:03.350018+0000","last_peered":"2024-04-26T20:26:03.350018+0000","last_clean":"2024-04-26T20:26:03.350018+0000","last_became_active":"2024-04-26T20:25:58.012072+0000","last_became_peered":"2024-04-26T20:25:58.012072+0000","last_unstale":"2024-04-26T20:26:03.350018+0000","last_undegraded":"2024-04-26T20:26:03.350018+0000","last_fullsized":"2024-04-26T20:26:03.350018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:10.380322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092736+0000","last_change":"2024-04-26T20:25:58.004587+0000","last_active":"2024-04-26T20:26:03.092736+0000","last_peered":"2024-04-26T20:26:03.092736+0000","last_clean":"2024-04-26T20:26:03.092736+0000","last_became_active":"2024-04-26T20:25:58.003068+0000","last_became_peered":"2024-04-26T20:25:58.003068+0000","last_unstale":"2024-04-26T20:26:03.092736+0000","last_undegraded":"2024-04-26T20:26:03.092736+0000","last_fullsized":"2024-04-26T20:26:03.092736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345668+0000","last_change":"2024-04-26T20:25:58.017953+0000","last_active":"2024-04-26T20:26:03.345668+0000","last_peered":"2024-04-26T20:26:03.345668+0000","last_clean":"2024-04-26T20:26:03.345668+0000","last_became_active":"2024-04-26T20:25:58.017321+0000","last_became_peered":"2024-04-26T20:25:58.017321+0000","last_unstale":"2024-04-26T20:26:03.345668+0000","last_undegraded":"2024-04-26T20:26:03.345668+0000","last_fullsized":"2024-04-26T20:26:03.345668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346846+0000","last_change":"2024-04-26T20:25:58.007941+0000","last_active":"2024-04-26T20:26:03.346846+0000","last_peered":"2024-04-26T20:26:03.346846+0000","last_clean":"2024-04-26T20:26:03.346846+0000","last_became_active":"2024-04-26T20:25:58.007043+0000","last_became_peered":"2024-04-26T20:25:58.007043+0000","last_unstale":"2024-04-26T20:26:03.346846+0000","last_undegraded":"2024-04-26T20:26:03.346846+0000","last_fullsized":"2024-04-26T20:26:03.346846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347334+0000","last_change":"2024-04-26T20:25:58.003034+0000","last_active":"2024-04-26T20:26:03.347334+0000","last_peered":"2024-04-26T20:26:03.347334+0000","last_clean":"2024-04-26T20:26:03.347334+0000","last_became_active":"2024-04-26T20:25:58.002792+0000","last_became_peered":"2024-04-26T20:25:58.002792+0000","last_unstale":"2024-04-26T20:26:03.347334+0000","last_undegraded":"2024-04-26T20:26:03.347334+0000","last_fullsized":"2024-04-26T20:26:03.347334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346106+0000","last_change":"2024-04-26T20:25:58.008698+0000","last_active":"2024-04-26T20:26:03.346106+0000","last_peered":"2024-04-26T20:26:03.346106+0000","last_clean":"2024-04-26T20:26:03.346106+0000","last_became_active":"2024-04-26T20:25:58.008279+0000","last_became_peered":"2024-04-26T20:25:58.008279+0000","last_unstale":"2024-04-26T20:26:03.346106+0000","last_undegraded":"2024-04-26T20:26:03.346106+0000","last_fullsized":"2024-04-26T20:26:03.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346355+0000","last_change":"2024-04-26T20:25:57.985508+0000","last_active":"2024-04-26T20:26:03.346355+0000","last_peered":"2024-04-26T20:26:03.346355+0000","last_clean":"2024-04-26T20:26:03.346355+0000","last_became_active":"2024-04-26T20:25:57.985227+0000","last_became_peered":"2024-04-26T20:25:57.985227+0000","last_unstale":"2024-04-26T20:26:03.346355+0000","last_undegraded":"2024-04-26T20:26:03.346355+0000","last_fullsized":"2024-04-26T20:26:03.346355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346661+0000","last_change":"2024-04-26T20:25:57.989552+0000","last_active":"2024-04-26T20:26:03.346661+0000","last_peered":"2024-04-26T20:26:03.346661+0000","last_clean":"2024-04-26T20:26:03.346661+0000","last_became_active":"2024-04-26T20:25:57.989141+0000","last_became_peered":"2024-04-26T20:25:57.989141+0000","last_unstale":"2024-04-26T20:26:03.346661+0000","last_undegraded":"2024-04-26T20:26:03.346661+0000","last_fullsized":"2024-04-26T20:26:03.346661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349524+0000","last_change":"2024-04-26T20:25:58.003504+0000","last_active":"2024-04-26T20:26:03.349524+0000","last_peered":"2024-04-26T20:26:03.349524+0000","last_clean":"2024-04-26T20:26:03.349524+0000","last_became_active":"2024-04-26T20:25:58.002880+0000","last_became_peered":"2024-04-26T20:25:58.002880+0000","last_unstale":"2024-04-26T20:26:03.349524+0000","last_undegraded":"2024-04-26T20:26:03.349524+0000","last_fullsized":"2024-04-26T20:26:03.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:47.751608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:27.133 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:26:27.133 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T20:26:27.133 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T20:26:27.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T20:26:27.134 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T20:26:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.366+0000 7fb086000640 1 Processor -- start 2024-04-26T20:26:27.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.367+0000 7fb086000640 1 -- start start 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.368+0000 7fb086000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0800d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.368+0000 7fb086000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0800d0480 con 0x7fb0800b5b80 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.368+0000 7fb086000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0800d0720 con 0x7fb0800d1620 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.368+0000 7fb086000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0800d5be0 con 0x7fb0800d5750 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.369+0000 7fb07fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb0800b5b80 0x7fb0800d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41370/0 (socket says 172.21.15.43:41370) 2024-04-26T20:26:27.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.369+0000 7fb07fe00640 1 -- 172.21.15.43:0/2367648377 learned_addr learned my addr 172.21.15.43:0/2367648377 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:27.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.369+0000 7fb085600640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0800d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.369+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2276652916 0 0) 0x7fb0800d0480 con 0x7fb0800b5b80 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.369+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb064003520 con 0x7fb0800b5b80 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2757893531 0 0) 0x7fb064003520 con 0x7fb0800b5b80 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 msgr2=0x7fb0800d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0800d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 >> v1:172.21.15.164:6789/0 conn(0x7fb0800d1620 legacy=0x7fb0800d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0800d0a00 con 0x7fb0800b5b80 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fb068001670 con 0x7fb0800b5b80 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 >> v1:172.21.15.43:6789/0 conn(0x7fb0800b5b80 legacy=0x7fb0800d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003f60 0x7fb064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800d5be0 con 0x7fb064004d40 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb06400bb50 con 0x7fb064003f60 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb06400bd50 con 0x7fb0800d5750 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb085600640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003f60 0x7fb064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.370+0000 7fb084c00640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.371+0000 7fb085600640 1 -- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 msgr2=0x7fb0640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.371+0000 7fb085600640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800d5750 0x7fb0640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.371+0000 7fb085600640 1 -- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003f60 msgr2=0x7fb064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.371+0000 7fb085600640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003f60 0x7fb064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.371+0000 7fb085600640 1 -- 172.21.15.43:0/2367648377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb064003520 con 0x7fb064004d40 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.372+0000 7fb085600640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ac0 tx=0x7fb064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a727e615bc483b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.372+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078036070 con 0x7fb064004d40 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.372+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb078031400 con 0x7fb064004d40 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.372+0000 7fb07f400640 1 -- 172.21.15.43:0/2367648377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078009690 con 0x7fb064004d40 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 -- 172.21.15.43:0/2367648377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 msgr2=0x7fb0640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 --2- 172.21.15.43:0/2367648377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ac0 tx=0x7fb064005830 comp rx=0 tx=0).stop 2024-04-26T20:26:27.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb085600640 1 -- 172.21.15.43:0/2367648377 reap_dead start 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 -- 172.21.15.43:0/2367648377 shutdown_connections 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 -- 172.21.15.43:0/2367648377 >> 172.21.15.43:0/2367648377 conn(0x7fb0800d4c30 msgr2=0x7fb0800d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 -- 172.21.15.43:0/2367648377 shutdown_connections 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.373+0000 7fb086000640 1 -- 172.21.15.43:0/2367648377 wait complete. 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.375+0000 7fb086000640 1 Processor -- start 2024-04-26T20:26:27.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.375+0000 7fb086000640 1 -- start start 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080155da0 0x7fb0800a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800a21b0 0x7fb0801f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800d2f10 con 0x7fb080155690 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800d0ce0 con 0x7fb080155da0 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb086000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0800d3090 con 0x7fb0800a21b0 2024-04-26T20:26:27.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb084c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb084c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54536/0 (socket says 172.21.15.43:54536) 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb084c00640 1 -- 172.21.15.43:0/2655195714 learned_addr learned my addr 172.21.15.43:0/2655195714 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.376+0000 7fb085600640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800a21b0 0x7fb0801f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080155da0 0x7fb0800a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 -- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800a21b0 msgr2=0x7fb0801f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800a21b0 0x7fb0801f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 -- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080155da0 msgr2=0x7fb0800a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080155da0 0x7fb0800a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 -- 172.21.15.43:0/2655195714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08009fa00 con 0x7fb080155690 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb084c00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e8c0 tx=0x7fb07000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070003b80 con 0x7fb080155690 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb070018070 con 0x7fb080155690 2024-04-26T20:26:27.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700050d0 con 0x7fb080155690 2024-04-26T20:26:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0800a0810 con 0x7fb080155690 2024-04-26T20:26:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.377+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0800a0ab0 con 0x7fb080155690 2024-04-26T20:26:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.379+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0800a25c0 con 0x7fb080155690 2024-04-26T20:26:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.379+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb070005270 con 0x7fb080155690 2024-04-26T20:26:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.380+0000 7fb07cc00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.381+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb07008bfd0 con 0x7fb080155690 2024-04-26T20:26:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.381+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.381+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 0x7fb05c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0801561a0 tx=0x7fb0680014f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.385+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0800a25c0 con 0x7fb080155690 2024-04-26T20:26:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.639+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb080069350 con 0x7fb080155690 2024-04-26T20:26:27.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.648+0000 7fb07cc00640 1 -- 172.21.15.43:0/2655195714 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174136 (secure 0 0 0) 0x7fb08009fa00 con 0x7fb080155690 2024-04-26T20:26:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:27.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 msgr2=0x7fb05c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 0x7fb05c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0801561a0 tx=0x7fb0680014f0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 msgr2=0x7fb08009f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e8c0 tx=0x7fb07000ed80 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 shutdown_connections 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb05c06c8f0 0x7fb05c06edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0800a21b0 0x7fb0801f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080155da0 0x7fb0800a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 --2- 172.21.15.43:0/2655195714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb080155690 0x7fb08009f4c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 >> 172.21.15.43:0/2655195714 conn(0x7fb0800d4c30 msgr2=0x7fb0800d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 shutdown_connections 2024-04-26T20:26:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.653+0000 7fb086000640 1 -- 172.21.15.43:0/2655195714 wait complete. 2024-04-26T20:26:27.674 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1074533441},{"type":"v1","addr":"172.21.15.43:6833","nonce":1074533441}]},"active_addr":"172.21.15.43:6833/1074533441","active_change":"2024-04-26T20:25:42.840258+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":344170179}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3009394128}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3311386228}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2163590327}]}]} 2024-04-26T20:26:27.677 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T20:26:27.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T20:26:27.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.888+0000 7f351d600640 1 Processor -- start 2024-04-26T20:26:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.889+0000 7f351d600640 1 -- start start 2024-04-26T20:26:27.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351d600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f35180cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351d600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3518072a80 con 0x7f35180ce2b0 2024-04-26T20:26:27.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351d600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35180cd560 con 0x7f35180cd180 2024-04-26T20:26:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351d600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3518072d00 con 0x7f3518072620 2024-04-26T20:26:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351cc00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f35180ce2b0 0x7f351806dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41374/0 (socket says 172.21.15.43:41374) 2024-04-26T20:26:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.890+0000 7f351cc00640 1 -- 172.21.15.43:0/749390266 learned_addr learned my addr 172.21.15.43:0/749390266 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f350fe00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f35180cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621771489 0 0) 0x7f3510001490 con 0x7f35180ce2b0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34fc003520 con 0x7f35180ce2b0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3352668756 0 0) 0x7f34fc003520 con 0x7f35180ce2b0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 msgr2=0x7f35180cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f35180cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 >> v1:172.21.15.164:6789/0 conn(0x7f35180cd180 legacy=0x7f351815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3518072fe0 con 0x7f35180ce2b0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f3510001bc0 con 0x7f35180ce2b0 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 >> v1:172.21.15.43:6789/0 conn(0x7f35180ce2b0 legacy=0x7f351806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34fc0039e0 0x7f34fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f34fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34fc004d40 0x7f34fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3518072d00 con 0x7f34fc0039e0 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc00ba70 con 0x7f34fc004d40 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.891+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f34fc00bc70 con 0x7f3518072620 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34fc0039e0 0x7f34fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f350fe00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f34fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 -- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 msgr2=0x7f34fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3518072620 0x7f34fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 -- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34fc004d40 msgr2=0x7f34fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f351cc00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34fc004d40 0x7f34fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34fc004d40 0x7f34fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f3517e00640 1 -- 172.21.15.43:0/749390266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc003520 con 0x7f34fc0039e0 2024-04-26T20:26:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.892+0000 7f351cc00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f34fc004d40 0x7f34fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.893+0000 7f3517e00640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34fc0039e0 0x7f34fc003e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3508001720 tx=0x7f34fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b618525f18394 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.893+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508001370 con 0x7f34fc0039e0 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.893+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35080052d0 con 0x7f34fc0039e0 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f3517400640 1 -- 172.21.15.43:0/749390266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350802abc0 con 0x7f34fc0039e0 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 -- 172.21.15.43:0/749390266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34fc0039e0 msgr2=0x7f34fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 --2- 172.21.15.43:0/749390266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34fc0039e0 0x7f34fc003e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3508001720 tx=0x7f34fc005830 comp rx=0 tx=0).stop 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351cc00640 1 -- 172.21.15.43:0/749390266 reap_dead start 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 -- 172.21.15.43:0/749390266 shutdown_connections 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 -- 172.21.15.43:0/749390266 >> 172.21.15.43:0/749390266 conn(0x7f3518068090 msgr2=0x7f3518068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 -- 172.21.15.43:0/749390266 shutdown_connections 2024-04-26T20:26:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.894+0000 7f351d600640 1 -- 172.21.15.43:0/749390266 wait complete. 2024-04-26T20:26:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.896+0000 7f351d600640 1 Processor -- start 2024-04-26T20:26:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.896+0000 7f351d600640 1 -- start start 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34fc0039e0 0x7f35180b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35180ac8b0 0x7f35180aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3518073da0 con 0x7f35180af0f0 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35180735a0 con 0x7f35180ac8b0 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351d600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f35180cdd70 con 0x7f34fc0039e0 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54554/0 (socket says 172.21.15.43:54554) 2024-04-26T20:26:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.897+0000 7f351cc00640 1 -- 172.21.15.43:0/3939984891 learned_addr learned my addr 172.21.15.43:0/3939984891 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f3517e00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34fc0039e0 0x7f35180b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f350fe00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35180ac8b0 0x7f35180aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 -- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34fc0039e0 msgr2=0x7f35180b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34fc0039e0 0x7f35180b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 -- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35180ac8b0 msgr2=0x7f35180aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35180ac8b0 0x7f35180aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 -- 172.21.15.43:0/3939984891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35180b5330 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351cc00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3510001750 tx=0x7f351000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000bb80 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351000bd20 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510007290 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35180afab0 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.898+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35180b4570 con 0x7f35180af0f0 2024-04-26T20:26:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.899+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3518146780 con 0x7f35180af0f0 2024-04-26T20:26:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.901+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f351001b060 con 0x7f35180af0f0 2024-04-26T20:26:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.902+0000 7f3514c00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 0x7f34f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.902+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f351008be90 con 0x7f35180af0f0 2024-04-26T20:26:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.902+0000 7f3517e00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 0x7f34f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.903+0000 7f3517e00640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 0x7f34f406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3508032000 tx=0x7f350802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:27.906+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3518146780 con 0x7f35180af0f0 2024-04-26T20:26:28.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.101+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3518069350 con 0x7f35180af0f0 2024-04-26T20:26:28.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.103+0000 7f3514c00640 1 -- 172.21.15.43:0/3939984891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f351008d020 con 0x7f35180af0f0 2024-04-26T20:26:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:28.105 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.105+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 msgr2=0x7f34f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.105+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 0x7f34f406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3508032000 tx=0x7f350802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 msgr2=0x7f35180af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3510001750 tx=0x7f351000b4a0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 shutdown_connections 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f34f406c920 0x7f34f406ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35180af0f0 0x7f35180af570 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f35180ac8b0 0x7f35180aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 --2- 172.21.15.43:0/3939984891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f34fc0039e0 0x7f35180b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 >> 172.21.15.43:0/3939984891 conn(0x7f3518068090 msgr2=0x7f35180b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 shutdown_connections 2024-04-26T20:26:28.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.106+0000 7f351d600640 1 -- 172.21.15.43:0/3939984891 wait complete. 2024-04-26T20:26:28.125 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T20:26:28.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.328+0000 7fecd1e00640 1 Processor -- start 2024-04-26T20:26:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.328+0000 7fecd1e00640 1 -- start start 2024-04-26T20:26:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.329+0000 7fecd1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feccc15f1b0 0x7feccc06de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7fecd1e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0b2640 con 0x7feccc1638a0 2024-04-26T20:26:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7fecd1e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc06e6c0 con 0x7feccc163380 2024-04-26T20:26:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7fecd1e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7feccc0b2f50 con 0x7feccc15f1b0 2024-04-26T20:26:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7fecca000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feccc1638a0 0x7feccc162c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41390/0 (socket says 172.21.15.43:41390) 2024-04-26T20:26:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7fecca000640 1 -- 172.21.15.43:0/1730197110 learned_addr learned my addr 172.21.15.43:0/1730197110 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.330+0000 7feccaa00640 1 --2- 172.21.15.43:0/1730197110 >> v2:172.21.15.164:3300/0 conn(0x7feccc15f1b0 0x7feccc06de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820008292 0 0) 0x7feccc0b2640 con 0x7feccc1638a0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecb0003520 con 0x7feccc1638a0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2370284397 0 0) 0x7fecb0003520 con 0x7feccc1638a0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 >> v2:172.21.15.164:3300/0 conn(0x7feccc15f1b0 msgr2=0x7feccc06de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 --2- 172.21.15.43:0/1730197110 >> v2:172.21.15.164:3300/0 conn(0x7feccc15f1b0 0x7feccc06de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 >> v1:172.21.15.164:6789/0 conn(0x7feccc163380 legacy=0x7feccc0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0b2900 con 0x7feccc1638a0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fecb4001670 con 0x7feccc1638a0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 >> v1:172.21.15.43:6789/0 conn(0x7feccc1638a0 legacy=0x7feccc162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00039b0 0x7fecb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecb0004800 0x7fecb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 0x7fecb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0b2f50 con 0x7fecb00051c0 2024-04-26T20:26:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecb000c060 con 0x7fecb0004800 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.331+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fecb000c260 con 0x7fecb00039b0 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 0x7fecb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccaa00640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00039b0 0x7fecb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7fecca000640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecb0004800 0x7fecb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 -- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00039b0 msgr2=0x7fecb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00039b0 0x7fecb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 -- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecb0004800 msgr2=0x7fecb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecb0004800 0x7fecb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.332+0000 7feccb400640 1 -- 172.21.15.43:0/1730197110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb0003520 con 0x7fecb00051c0 2024-04-26T20:26:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.333+0000 7feccb400640 1 -- 172.21.15.43:0/1730197110 reap_dead start 2024-04-26T20:26:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.333+0000 7feccb400640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 0x7fecb000b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001e20 tx=0x7fecb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4985da8cbfba65a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.333+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4001a70 con 0x7fecb00051c0 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc402ae40 con 0x7fecb00051c0 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecc9600640 1 -- 172.21.15.43:0/1730197110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4003bb0 con 0x7fecb00051c0 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 -- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 msgr2=0x7fecb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 0x7fecb000b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001e20 tx=0x7fecb0005830 comp rx=0 tx=0).stop 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 -- 172.21.15.43:0/1730197110 shutdown_connections 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 --2- 172.21.15.43:0/1730197110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecb00051c0 0x7fecb000b9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 -- 172.21.15.43:0/1730197110 >> 172.21.15.43:0/1730197110 conn(0x7feccc0b47f0 msgr2=0x7feccc0b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 -- 172.21.15.43:0/1730197110 shutdown_connections 2024-04-26T20:26:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.334+0000 7fecd1e00640 1 -- 172.21.15.43:0/1730197110 wait complete. 2024-04-26T20:26:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.336+0000 7fecd1e00640 1 Processor -- start 2024-04-26T20:26:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.336+0000 7fecd1e00640 1 -- start start 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 0x7feccc0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 0x7feccc161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feccc162c70 con 0x7feccc15fb90 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feccc15f590 con 0x7feccc15f710 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7fecd1e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feccc162df0 con 0x7fecb00051c0 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7feccb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7feccb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54574/0 (socket says 172.21.15.43:54574) 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.337+0000 7feccb400640 1 -- 172.21.15.43:0/2687049021 learned_addr learned my addr 172.21.15.43:0/2687049021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7fecca000640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 0x7feccc161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccaa00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 0x7feccc0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 -- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 msgr2=0x7feccc0b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 0x7feccc0b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 -- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 msgr2=0x7feccc161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 0x7feccc161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 -- 172.21.15.43:0/2687049021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0c81c0 con 0x7feccc15fb90 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccaa00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 0x7feccc0b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7fecca000640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 0x7feccc161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7feccb400640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001f10 tx=0x7fecc40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007db0 con 0x7feccc15fb90 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.338+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc4001250 con 0x7feccc15fb90 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.339+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4006b50 con 0x7feccc15fb90 2024-04-26T20:26:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.339+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc162220 con 0x7feccc15fb90 2024-04-26T20:26:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.339+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc0afc90 con 0x7feccc15fb90 2024-04-26T20:26:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.340+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc15ceb0 con 0x7feccc15fb90 2024-04-26T20:26:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.341+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecc4006cf0 con 0x7feccc15fb90 2024-04-26T20:26:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.342+0000 7fecc3400640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 0x7feca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.342+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fecc40ab2a0 con 0x7feccc15fb90 2024-04-26T20:26:28.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.342+0000 7feccaa00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 0x7feca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.343+0000 7feccaa00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 0x7feca806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feccc1f8b40 tx=0x7fecbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.346+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feccc15ceb0 con 0x7feccc15fb90 2024-04-26T20:26:28.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.551+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feccc069a60 con 0x7feccc15fb90 2024-04-26T20:26:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.554+0000 7fecc3400640 1 -- 172.21.15.43:0/2687049021 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7fecc407f030 con 0x7feccc15fb90 2024-04-26T20:26:28.556 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:28.556 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 msgr2=0x7feca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 0x7feca806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feccc1f8b40 tx=0x7fecbc008040 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 msgr2=0x7feccc161850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001f10 tx=0x7fecc40014a0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 shutdown_connections 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7feca806c8f0 0x7feca806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feccc15fb90 0x7feccc161850 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feccc15f710 0x7feccc161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 --2- 172.21.15.43:0/2687049021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fecb00051c0 0x7feccc0b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 >> 172.21.15.43:0/2687049021 conn(0x7feccc0b47f0 msgr2=0x7feccc070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 shutdown_connections 2024-04-26T20:26:28.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.557+0000 7fecd1e00640 1 -- 172.21.15.43:0/2687049021 wait complete. 2024-04-26T20:26:28.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T20:26:28.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T20:26:28.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T20:26:28.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T20:26:28.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T20:26:28.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T20:26:28.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T20:26:28.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T20:26:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.880+0000 7fa875600640 1 Processor -- start 2024-04-26T20:26:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.881+0000 7fa875600640 1 -- start start 2024-04-26T20:26:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.880+0000 7ff0ca200640 1 Processor -- start 2024-04-26T20:26:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.880+0000 7ff0ca200640 1 -- start start 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7fa875600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7fa875600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa8700b4200 0x7fa87006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7fa875600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa87006ea10 con 0x7fa87006e220 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7fa875600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa8700b4660 con 0x7fa8700b4200 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7fa875600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa870141920 con 0x7fa870141540 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7ff0ca200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0c41630a0 0x7ff0c4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7ff0ca200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0c4163480 0x7ff0c40d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7f0433c00640 1 Processor -- start 2024-04-26T20:26:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.883+0000 7ff0ca200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c40736f0 con 0x7ff0c40d0da0 2024-04-26T20:26:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55260/0 (socket says 172.21.15.43:55260) 2024-04-26T20:26:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7fa86fe00640 1 -- 172.21.15.43:0/4062203621 learned_addr learned my addr 172.21.15.43:0/4062203621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7f0433c00640 1 -- start start 2024-04-26T20:26:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7ff0ca200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c40d11f0 con 0x7ff0c41630a0 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7ff0ca200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c406e7a0 con 0x7ff0c4163480 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7ff0c3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0c4163480 0x7ff0c40d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7ff0c3e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0c4163480 0x7ff0c40d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46322/0 (socket says 172.21.15.43:46322) 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.884+0000 7fa86f400640 1 --2- 172.21.15.43:0/4062203621 >> v2:172.21.15.164:3300/0 conn(0x7fa8700b4200 0x7fa87006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7fa86fe00640 1 -- 172.21.15.43:0/4062203621 >> v2:172.21.15.164:3300/0 conn(0x7fa8700b4200 msgr2=0x7fa87006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7fa86fe00640 1 --2- 172.21.15.43:0/4062203621 >> v2:172.21.15.164:3300/0 conn(0x7fa8700b4200 0x7fa87006dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7fa86fe00640 1 -- 172.21.15.43:0/4062203621 >> v1:172.21.15.164:6789/0 conn(0x7fa87006e220 legacy=0x7fa87015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7fa86fe00640 1 -- 172.21.15.43:0/4062203621 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87006ecf0 con 0x7fa870141540 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7ff0c9800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff0c40d0da0 0x7ff0c4166d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41400/0 (socket says 172.21.15.43:41400) 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.885+0000 7ff0c9800640 1 -- 172.21.15.43:0/3119710437 learned_addr learned my addr 172.21.15.43:0/3119710437 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.886+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3101789186 0 0) 0x7ff0b4001110 con 0x7ff0c40d0da0 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.886+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0a8003520 con 0x7ff0c40d0da0 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.886+0000 7ff0c8e00640 1 --2- 172.21.15.43:0/3119710437 >> v2:172.21.15.43:3300/0 conn(0x7ff0c41630a0 0x7ff0c4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3346393117 0 0) 0x7ff0a8003520 con 0x7ff0c40d0da0 2024-04-26T20:26:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 >> v2:172.21.15.164:3300/0 conn(0x7ff0c4163480 msgr2=0x7ff0c40d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 --2- 172.21.15.43:0/3119710437 >> v2:172.21.15.164:3300/0 conn(0x7ff0c4163480 0x7ff0c40d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 >> v2:172.21.15.43:3300/0 conn(0x7ff0c41630a0 msgr2=0x7ff0c4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7f0433c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f042c15e6c0 0x7f042c072f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7fa86fe00640 1 --2- 172.21.15.43:0/4062203621 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009870 tx=0x7fa85c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0510e23d1b35ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7fa86ea00640 1 -- 172.21.15.43:0/4062203621 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c037070 con 0x7fa870141540 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7fa86ea00640 1 -- 172.21.15.43:0/4062203621 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c032440 con 0x7fa870141540 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7fa86ea00640 1 -- 172.21.15.43:0/4062203621 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c032810 con 0x7fa870141540 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 --2- 172.21.15.43:0/3119710437 >> v2:172.21.15.43:3300/0 conn(0x7ff0c41630a0 0x7ff0c4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c406ea60 con 0x7ff0c40d0da0 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7ff0b4001840 con 0x7ff0c40d0da0 2024-04-26T20:26:28.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 >> v1:172.21.15.43:6789/0 conn(0x7ff0c40d0da0 legacy=0x7ff0c4166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a80039b0 0x7ff0a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8005210 0x7ff0a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c40d11f0 con 0x7ff0a8005210 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c406e7a0 con 0x7ff0a80039b0 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a800c1c0 con 0x7ff0a8004800 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7ff0c8e00640 1 --2- 172.21.15.43:0/3119710437 >> v2:172.21.15.43:3300/0 conn(0x7ff0c41630a0 0x7ff0c4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:28.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.887+0000 7f536f800640 1 Processor -- start 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7ff0c9800640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8005210 0x7ff0a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7ff0c8e00640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a80039b0 0x7ff0a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.888+0000 7f536f800640 1 -- start start 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7fa875600640 1 -- 172.21.15.43:0/4062203621 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 msgr2=0x7fa8700b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7fa875600640 1 --2- 172.21.15.43:0/4062203621 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009870 tx=0x7fa85c02a270 comp rx=0 tx=0).stop 2024-04-26T20:26:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7f0433c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c071ea0 con 0x7f042c0b2ec0 2024-04-26T20:26:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7f0433c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c072180 con 0x7f042c073480 2024-04-26T20:26:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.889+0000 7f0433c00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f042c072460 con 0x7f042c15e6c0 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7ff0c3e00640 1 -- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a80039b0 msgr2=0x7ff0a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0a80039b0 0x7ff0a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7ff0c3e00640 1 -- 172.21.15.43:0/3119710437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8005210 msgr2=0x7ff0a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0a8005210 0x7ff0a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7ff0c3e00640 1 -- 172.21.15.43:0/3119710437 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c406ea60 con 0x7ff0a8004800 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7fa875600640 1 -- 172.21.15.43:0/4062203621 shutdown_connections 2024-04-26T20:26:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7fa875600640 1 --2- 172.21.15.43:0/4062203621 >> v2:172.21.15.164:3300/0 conn(0x7fa8700b4200 0x7fa87006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7f0431200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f042c15e6c0 0x7f042c072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7fa875600640 1 --2- 172.21.15.43:0/4062203621 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa8700b3cc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7fa875600640 1 -- 172.21.15.43:0/4062203621 >> 172.21.15.43:0/4062203621 conn(0x7fa8700b26b0 msgr2=0x7fa8700b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7f536f800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 0x7f5368072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.890+0000 7f536f800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f7771200640 1 Processor -- start 2024-04-26T20:26:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f536f800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53680733c0 con 0x7f5368072e40 2024-04-26T20:26:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f536f800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f536814d170 con 0x7f536814cd90 2024-04-26T20:26:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f536f800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f536806e330 con 0x7f53681639c0 2024-04-26T20:26:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7ff0c9800640 1 -- 172.21.15.43:0/3119710437 reap_dead start 2024-04-26T20:26:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7fa875600640 1 -- 172.21.15.43:0/4062203621 shutdown_connections 2024-04-26T20:26:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f042be00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f042c073480 0x7f042c141fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:44140/0 (socket says 172.21.15.43:44140) 2024-04-26T20:26:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f042be00640 1 -- 172.21.15.43:0/3535269768 learned_addr learned my addr 172.21.15.43:0/3535269768 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f7771200640 1 -- start start 2024-04-26T20:26:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f536ce00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 0x7f5368072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f0431200640 1 -- 172.21.15.43:0/3535269768 >> v1:172.21.15.164:6789/0 conn(0x7f042c073480 legacy=0x7f042c141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7f0431200640 1 -- 172.21.15.43:0/3535269768 >> v1:172.21.15.43:6789/0 conn(0x7f042c0b2ec0 legacy=0x7f042c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f5367e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f6d89400640 1 Processor -- start 2024-04-26T20:26:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f536d800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5368072e40 0x7f5368166f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:44148/0 (socket says 172.21.15.43:44148) 2024-04-26T20:26:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f536d800640 1 -- 172.21.15.43:0/1531589874 learned_addr learned my addr 172.21.15.43:0/1531589874 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0a8004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000ba30 tx=0x7ff0a8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d43e224925a687f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f6d89400640 1 -- start start 2024-04-26T20:26:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.891+0000 7fa875600640 1 -- 172.21.15.43:0/4062203621 wait complete. 2024-04-26T20:26:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367400640 1 -- 172.21.15.43:0/1531589874 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876166383 0 0) 0x7f53680733c0 con 0x7f5368072e40 2024-04-26T20:26:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0030070 con 0x7ff0a8004800 2024-04-26T20:26:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367400640 1 -- 172.21.15.43:0/1531589874 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5350003520 con 0x7f5368072e40 2024-04-26T20:26:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.892+0000 7f0431200640 1 -- 172.21.15.43:0/3535269768 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c072740 con 0x7f042c15e6c0 2024-04-26T20:26:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7fa875600640 1 Processor -- start 2024-04-26T20:26:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7ff0c3400640 1 -- 172.21.15.43:0/3119710437 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b002add0 con 0x7ff0a8004800 2024-04-26T20:26:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f7771200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f776c156000 0x7f776c160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f7771200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f776c06ed20 0x7f776c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f7771200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776c072fb0 con 0x7f776c15ccc0 2024-04-26T20:26:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f7771200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f776c06f6d0 con 0x7f776c156000 2024-04-26T20:26:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f7771200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f776c072d70 con 0x7f776c06ed20 2024-04-26T20:26:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f776b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f776c156000 0x7f776c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367e00640 1 -- 172.21.15.43:0/1531589874 >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 msgr2=0x7f5368072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367e00640 1 --2- 172.21.15.43:0/1531589874 >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 0x7f5368072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367e00640 1 -- 172.21.15.43:0/1531589874 >> v1:172.21.15.164:6789/0 conn(0x7f5368072e40 legacy=0x7f5368166f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.893+0000 7f5367e00640 1 -- 172.21.15.43:0/1531589874 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53680736a0 con 0x7f53681639c0 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f536ce00640 1 --2- 172.21.15.43:0/1531589874 >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 0x7f5368072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7fa875600640 1 -- start start 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7ff0ca200640 1 -- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 msgr2=0x7ff0a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7ff0ca200640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0a8004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000ba30 tx=0x7ff0a8005830 comp rx=0 tx=0).stop 2024-04-26T20:26:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f0431200640 1 --2- 172.21.15.43:0/3535269768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c15e6c0 0x7f042c072f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f041c009a00 tx=0x7f041c02a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8a34e6ad07fe5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f6d89400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6d841634a0 0x7f6d840737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d89400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d840b43f0 con 0x7f6d840d0240 2024-04-26T20:26:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f776aa00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f776c15ccc0 0x7f776c072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41412/0 (socket says 172.21.15.43:41412) 2024-04-26T20:26:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.894+0000 7f776aa00640 1 -- 172.21.15.43:0/3644102313 learned_addr learned my addr 172.21.15.43:0/3644102313 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f776be00640 1 --2- 172.21.15.43:0/3644102313 >> v2:172.21.15.164:3300/0 conn(0x7f776c06ed20 0x7f776c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d89400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d84073e20 con 0x7f6d841630c0 2024-04-26T20:26:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d89400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6d840d06b0 con 0x7f6d841634a0 2024-04-26T20:26:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7ff0ca200640 1 -- 172.21.15.43:0/3119710437 shutdown_connections 2024-04-26T20:26:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f5367e00640 1 --2- 172.21.15.43:0/1531589874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b39b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53540097a0 tx=0x7f535402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340e3a2153dcaae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7ff0ca200640 1 --2- 172.21.15.43:0/3119710437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0a8004c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d88a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6d840d0240 0x7f6d840af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41428/0 (socket says 172.21.15.43:41428) 2024-04-26T20:26:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d88a00640 1 -- 172.21.15.43:0/740369263 learned_addr learned my addr 172.21.15.43:0/740369263 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7ff0ca200640 1 -- 172.21.15.43:0/3119710437 >> 172.21.15.43:0/3119710437 conn(0x7ff0c4068090 msgr2=0x7ff0c4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.895+0000 7f6d83400640 1 --2- 172.21.15.43:0/740369263 >> v2:172.21.15.164:3300/0 conn(0x7f6d841634a0 0x7f6d840737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7ff0ca200640 1 -- 172.21.15.43:0/3119710437 shutdown_connections 2024-04-26T20:26:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f042b400640 1 -- 172.21.15.43:0/3535269768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c036070 con 0x7f042c15e6c0 2024-04-26T20:26:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f5367400640 1 -- 172.21.15.43:0/1531589874 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354037070 con 0x7f53681639c0 2024-04-26T20:26:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f5367400640 1 -- 172.21.15.43:0/1531589874 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5354032440 con 0x7f53681639c0 2024-04-26T20:26:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f5367400640 1 -- 172.21.15.43:0/1531589874 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354032850 con 0x7f53681639c0 2024-04-26T20:26:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f776be00640 1 -- 172.21.15.43:0/3644102313 >> v2:172.21.15.43:3300/0 conn(0x7f776c156000 msgr2=0x7f776c160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f776be00640 1 --2- 172.21.15.43:0/3644102313 >> v2:172.21.15.43:3300/0 conn(0x7f776c156000 0x7f776c160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f776be00640 1 -- 172.21.15.43:0/3644102313 >> v1:172.21.15.43:6789/0 conn(0x7f776c15ccc0 legacy=0x7f776c072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f776be00640 1 -- 172.21.15.43:0/3644102313 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c15f640 con 0x7f776c06ed20 2024-04-26T20:26:28.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7f042b400640 1 -- 172.21.15.43:0/3535269768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c031600 con 0x7f042c15e6c0 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7fa875600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 0x7fa8701680a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7fa875600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8700b4200 0x7fa8701368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7fa875600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa870143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7fa875600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa87006f590 con 0x7fa8700b4200 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7fa875600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa87006f2b0 con 0x7fa87006e220 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7fa875600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa87006e630 con 0x7fa870141540 2024-04-26T20:26:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806346328 0 0) 0x7f6d84073e20 con 0x7f6d841630c0 2024-04-26T20:26:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d68003520 con 0x7f6d841630c0 2024-04-26T20:26:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f536f800640 1 -- 172.21.15.43:0/1531589874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 msgr2=0x7f53680b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f536f800640 1 --2- 172.21.15.43:0/1531589874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b39b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53540097a0 tx=0x7f535402a130 comp rx=0 tx=0).stop 2024-04-26T20:26:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f042b400640 1 -- 172.21.15.43:0/3535269768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c031a10 con 0x7f042c15e6c0 2024-04-26T20:26:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.896+0000 7ff0ca200640 1 -- 172.21.15.43:0/3119710437 wait complete. 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.897+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332859311 0 0) 0x7f6d840b43f0 con 0x7f6d840d0240 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d84073e20 con 0x7f6d840d0240 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 0x7fa8701680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7fa86fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 0x7fa8701680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46370/0 (socket says 172.21.15.43:46370) 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7fa86fe00640 1 -- 172.21.15.43:0/3389369987 learned_addr learned my addr 172.21.15.43:0/3389369987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7fa874c00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa870143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f776be00640 1 --2- 172.21.15.43:0/3644102313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c06f190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f776c06d460 tx=0x7f776402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37da64e8e6c522f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f6d83400640 1 -- 172.21.15.43:0/740369263 >> v1:172.21.15.164:6789/0 conn(0x7f6d841630c0 legacy=0x7f6d840b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f6d83400640 1 -- 172.21.15.43:0/740369263 >> v1:172.21.15.43:6789/0 conn(0x7f6d840d0240 legacy=0x7f6d840af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f6d83400640 1 -- 172.21.15.43:0/740369263 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d84071c50 con 0x7f6d841634a0 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f536f800640 1 -- 172.21.15.43:0/1531589874 shutdown_connections 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f536f800640 1 --2- 172.21.15.43:0/1531589874 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b39b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f536f800640 1 --2- 172.21.15.43:0/1531589874 >> v2:172.21.15.164:3300/0 conn(0x7f536814cd90 0x7f5368072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f536f800640 1 -- 172.21.15.43:0/1531589874 >> 172.21.15.43:0/1531589874 conn(0x7f53680b26b0 msgr2=0x7f53680b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7fa86f400640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8700b4200 0x7fa8701368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f0433c00640 1 -- 172.21.15.43:0/3535269768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c15e6c0 msgr2=0x7f042c072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f0433c00640 1 --2- 172.21.15.43:0/3535269768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c15e6c0 0x7f042c072f40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f041c009a00 tx=0x7f041c02a450 comp rx=0 tx=0).stop 2024-04-26T20:26:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f536f800640 1 -- 172.21.15.43:0/1531589874 shutdown_connections 2024-04-26T20:26:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f776a000640 1 -- 172.21.15.43:0/3644102313 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764037070 con 0x7f776c06ed20 2024-04-26T20:26:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f776a000640 1 -- 172.21.15.43:0/3644102313 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764032440 con 0x7f776c06ed20 2024-04-26T20:26:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.898+0000 7f776a000640 1 -- 172.21.15.43:0/3644102313 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776402ae60 con 0x7f776c06ed20 2024-04-26T20:26:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7ff0ca200640 1 Processor -- start 2024-04-26T20:26:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f0433c00640 1 -- 172.21.15.43:0/3535269768 shutdown_connections 2024-04-26T20:26:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f0433c00640 1 --2- 172.21.15.43:0/3535269768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c15e6c0 0x7f042c072f40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f0433c00640 1 -- 172.21.15.43:0/3535269768 >> 172.21.15.43:0/3535269768 conn(0x7f042c068940 msgr2=0x7f042c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7ff0ca200640 1 -- start start 2024-04-26T20:26:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f7771200640 1 -- 172.21.15.43:0/3644102313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 msgr2=0x7f776c06f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f7771200640 1 --2- 172.21.15.43:0/3644102313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c06f190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f776c06d460 tx=0x7f776402a110 comp rx=0 tx=0).stop 2024-04-26T20:26:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f6d83400640 1 --2- 172.21.15.43:0/740369263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d841634a0 0x7f6d840737b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d700097a0 tx=0x7f6d7002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78116a771e856ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f0433c00640 1 -- 172.21.15.43:0/3535269768 shutdown_connections 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 -- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 msgr2=0x7fa870143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa870143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 -- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 msgr2=0x7fa8701680a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 0x7fa8701680a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa870144b30 con 0x7fa8700b4200 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa874c00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa870141540 0x7fa870143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86fe00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa87006e220 0x7fa8701680a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7fa86f400640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8700b4200 0x7fa8701368e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa86000d5c0 tx=0x7fa86000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7ff0ca200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0c4158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7ff0ca200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0c4146b50 0x7ff0c4160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7ff0ca200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 0x7ff0c4161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0c4146b50 0x7ff0c4160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7ff0c3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0c4146b50 0x7ff0c4160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46386/0 (socket says 172.21.15.43:46386) 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7ff0c8e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0c4158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7ff0c8e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0c4158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55292/0 (socket says 172.21.15.43:55292) 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70037070 con 0x7f6d841634a0 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d70032440 con 0x7f6d841634a0 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f6d82a00640 1 -- 172.21.15.43:0/740369263 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70032810 con 0x7f6d841634a0 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7f6d89400640 1 -- 172.21.15.43:0/740369263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d841634a0 msgr2=0x7f6d840737b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7f6d89400640 1 --2- 172.21.15.43:0/740369263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d841634a0 0x7f6d840737b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d700097a0 tx=0x7f6d7002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f6d89400640 1 -- 172.21.15.43:0/740369263 shutdown_connections 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f6d89400640 1 --2- 172.21.15.43:0/740369263 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d841634a0 0x7f6d840737b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f6d89400640 1 -- 172.21.15.43:0/740369263 >> 172.21.15.43:0/740369263 conn(0x7f6d840b2640 msgr2=0x7f6d840b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f6d89400640 1 -- 172.21.15.43:0/740369263 shutdown_connections 2024-04-26T20:26:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f6d89400640 1 -- 172.21.15.43:0/740369263 wait complete. 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.899+0000 7f536f800640 1 -- 172.21.15.43:0/1531589874 wait complete. 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f536f800640 1 Processor -- start 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 -- 172.21.15.43:0/3644102313 shutdown_connections 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 --2- 172.21.15.43:0/3644102313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c06f190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 --2- 172.21.15.43:0/3644102313 >> v2:172.21.15.43:3300/0 conn(0x7f776c156000 0x7f776c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 -- 172.21.15.43:0/3644102313 >> 172.21.15.43:0/3644102313 conn(0x7f776c068090 msgr2=0x7f776c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 -- 172.21.15.43:0/3644102313 shutdown_connections 2024-04-26T20:26:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f7771200640 1 -- 172.21.15.43:0/3644102313 wait complete. 2024-04-26T20:26:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7fb798a00640 1 Processor -- start 2024-04-26T20:26:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.901+0000 7fb798a00640 1 -- start start 2024-04-26T20:26:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600125f0 con 0x7fa8700b4200 2024-04-26T20:26:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.900+0000 7f0433c00640 1 -- 172.21.15.43:0/3535269768 wait complete. 2024-04-26T20:26:28.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860019070 con 0x7fa8700b4200 2024-04-26T20:26:28.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600129b0 con 0x7fa8700b4200 2024-04-26T20:26:28.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7ff0c9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 0x7ff0c4161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7ff0c9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 0x7ff0c4161ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54602/0 (socket says 172.21.15.43:54602) 2024-04-26T20:26:28.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f0433c00640 1 Processor -- start 2024-04-26T20:26:28.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7ff0c3e00640 1 -- 172.21.15.43:0/1273316865 learned_addr learned my addr 172.21.15.43:0/1273316865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c40b58e0 con 0x7ff0c4161640 2024-04-26T20:26:28.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c4073c50 con 0x7ff0c4146b50 2024-04-26T20:26:28.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8701420b0 con 0x7fa8700b4200 2024-04-26T20:26:28.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87013a920 con 0x7fa8700b4200 2024-04-26T20:26:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7f7771200640 1 Processor -- start 2024-04-26T20:26:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7f0433c00640 1 -- start start 2024-04-26T20:26:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fb798a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb794073300 0x7fb79406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fb798a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79406ecf0 con 0x7fb7940b3cc0 2024-04-26T20:26:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fb798a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79406efd0 con 0x7fb7940b31d0 2024-04-26T20:26:28.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.903+0000 7fb798a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb794161c70 con 0x7fb794073300 2024-04-26T20:26:28.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c40b5a60 con 0x7ff0a8004800 2024-04-26T20:26:28.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f273a400640 1 Processor -- start 2024-04-26T20:26:28.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f7771200640 1 -- start start 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.902+0000 7f536f800640 1 -- start start 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5368072e40 0x7f53681fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 0x7f53680b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53680b5d10 con 0x7f536814cd90 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f536806dfc0 con 0x7f5368072e40 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f536f800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53680b3ef0 con 0x7f53681639c0 2024-04-26T20:26:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7f6d89400640 1 Processor -- start 2024-04-26T20:26:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7f273a400640 1 -- start start 2024-04-26T20:26:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7fb793400640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb7940b31d0 0x7fb7941412d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:44160/0 (socket says 172.21.15.43:44160) 2024-04-26T20:26:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.904+0000 7fb793400640 1 -- 172.21.15.43:0/1225660911 learned_addr learned my addr 172.21.15.43:0/1225660911 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7f6d89400640 1 -- start start 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7ff0c9800640 1 -- 172.21.15.43:0/1273316865 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 msgr2=0x7ff0c4158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7ff0c9800640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0a8004800 0x7ff0c4158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7ff0c9800640 1 -- 172.21.15.43:0/1273316865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0c4146b50 msgr2=0x7ff0c4160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7f5367e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 0x7f53680b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7ff0c9800640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0c4146b50 0x7ff0c4160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7ff0c9800640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c4160790 con 0x7ff0c4161640 2024-04-26T20:26:28.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7f5367e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 0x7f53680b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54618/0 (socket says 172.21.15.43:54618) 2024-04-26T20:26:28.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7f5367e00640 1 -- 172.21.15.43:0/1890516861 learned_addr learned my addr 172.21.15.43:0/1890516861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa870069350 con 0x7fa8700b4200 2024-04-26T20:26:28.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f536d800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7ff0c9800640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 0x7ff0c4161ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4163910 tx=0x7ff0b4007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f5367e00640 1 -- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 msgr2=0x7f53680b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f5367e00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53681639c0 0x7f53680b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f5367e00640 1 -- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5368072e40 msgr2=0x7f53681fcd00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f5367e00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5368072e40 0x7f53681fcd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f5367e00640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681eb070 con 0x7f536814cd90 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f273a400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f273415e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f273a400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2734073380 0x7f273415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f273a400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2734072180 con 0x7f2734072f40 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f273a400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f27340737e0 con 0x7f2734073380 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7f273a400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2734072460 con 0x7f27340b2ec0 2024-04-26T20:26:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d840d0240 0x7f6d8415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.905+0000 7fb793e00640 1 --2- 172.21.15.43:0/1225660911 >> v2:172.21.15.164:3300/0 conn(0x7fb794073300 0x7fb79406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7fb793e00640 1 -- 172.21.15.43:0/1225660911 >> v1:172.21.15.164:6789/0 conn(0x7fb7940b31d0 legacy=0x7fb7941412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f2733e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2734073380 0x7f273415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7fb793e00640 1 -- 172.21.15.43:0/1225660911 >> v1:172.21.15.43:6789/0 conn(0x7fb7940b3cc0 legacy=0x7fb79406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.906+0000 7fb793e00640 1 -- 172.21.15.43:0/1225660911 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79406f2b0 con 0x7fb794073300 2024-04-26T20:26:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f2733e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2734073380 0x7f273415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46416/0 (socket says 172.21.15.43:46416) 2024-04-26T20:26:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f2733e00640 1 -- 172.21.15.43:0/991437659 learned_addr learned my addr 172.21.15.43:0/991437659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d841630c0 0x7f6d84164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 0x7f6d841ed990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b401a070 con 0x7ff0c4161640 2024-04-26T20:26:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f5367e00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 0x7f53680b1910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f535402a720 tx=0x7f53540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f2733400640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f273415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b400f8f0 con 0x7ff0c4161640 2024-04-26T20:26:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6d84072210 con 0x7f6d841634a0 2024-04-26T20:26:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840b5b50 con 0x7f6d840d0240 2024-04-26T20:26:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d89400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6d84072390 con 0x7f6d841630c0 2024-04-26T20:26:28.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400fcc0 con 0x7ff0c4161640 2024-04-26T20:26:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f6d88a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 0x7f6d841ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7f7771200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c41612b0 con 0x7ff0c4161640 2024-04-26T20:26:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.907+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c4160a10 con 0x7ff0c4161640 2024-04-26T20:26:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f7771200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776c156000 0x7f776c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f6d88a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 0x7f6d841ed990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54630/0 (socket says 172.21.15.43:54630) 2024-04-26T20:26:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354037050 con 0x7f536814cd90 2024-04-26T20:26:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5354043070 con 0x7f536814cd90 2024-04-26T20:26:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535403cb10 con 0x7f536814cd90 2024-04-26T20:26:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f7771200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 0x7f776c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f6d88a00640 1 -- 172.21.15.43:0/4201253410 learned_addr learned my addr 172.21.15.43:0/4201253410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f2733e00640 1 -- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 msgr2=0x7f273415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f2733e00640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f273415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f2733e00640 1 -- 172.21.15.43:0/991437659 >> v1:172.21.15.164:6789/0 conn(0x7f2734072f40 legacy=0x7f2734141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f2733e00640 1 -- 172.21.15.43:0/991437659 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734072740 con 0x7f2734073380 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681eb350 con 0x7f536814cd90 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681eb910 con 0x7f536814cd90 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f7771200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1563e0 con 0x7f776c15ccc0 2024-04-26T20:26:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f7771200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f776c15fc00 con 0x7f776c06ed20 2024-04-26T20:26:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f7771200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f776c073290 con 0x7f776c156000 2024-04-26T20:26:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7f6d83e00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d840d0240 0x7f6d8415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f776be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 0x7f776c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f776aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776c156000 0x7f776c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f776be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 0x7f776c153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54636/0 (socket says 172.21.15.43:54636) 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f776be00640 1 -- 172.21.15.43:0/79013580 learned_addr learned my addr 172.21.15.43:0/79013580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f6d88a00640 1 -- 172.21.15.43:0/4201253410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d841630c0 msgr2=0x7f6d84164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f6d88a00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d841630c0 0x7f6d84164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f6d88a00640 1 -- 172.21.15.43:0/4201253410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d840d0240 msgr2=0x7f6d8415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f6d88a00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d840d0240 0x7f6d8415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f6d88a00640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d84165210 con 0x7f6d841634a0 2024-04-26T20:26:28.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.908+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa860012b50 con 0x7fa8700b4200 2024-04-26T20:26:28.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7f776b400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0c4069350 con 0x7ff0c4161640 2024-04-26T20:26:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.909+0000 7fa867e00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa85006c940 0x7fa85006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f0433c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 0x7f042c1f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 -- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776c156000 msgr2=0x7f776c152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776c156000 0x7f776c152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 -- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 msgr2=0x7f776c13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c142280 con 0x7f776c15ccc0 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fb793e00640 1 --2- 172.21.15.43:0/1225660911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb794073300 0x7fb79406e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdac425c19c3a60d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa86008c1e0 con 0x7fa8700b4200 2024-04-26T20:26:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fa86fe00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa85006c940 0x7fa85006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776b400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776c06ed20 0x7f776c13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fa867e00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7fa8500760b0 0x7fa8500784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa860001d30 con 0x7fa8500760b0 2024-04-26T20:26:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa860093050 con 0x7fa8700b4200 2024-04-26T20:26:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f776be00640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 0x7f776c153500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f776402a700 tx=0x7f77640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7fa874c00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7fa8500760b0 0x7fa8500784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f0433c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c0b2ec0 0x7f042c1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0433c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042c15e6c0 0x7f042c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7f6d88a00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 0x7f6d841ed990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00cb40 tx=0x7f6d7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fb792000640 1 -- 172.21.15.43:0/1225660911 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c007740 con 0x7fb794073300 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fb792000640 1 -- 172.21.15.43:0/1225660911 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c0078e0 con 0x7fb794073300 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.910+0000 7fb792000640 1 -- 172.21.15.43:0/1225660911 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c007cb0 con 0x7fb794073300 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0433c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1426d0 con 0x7f042c0b2ec0 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0433c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f042c15eaa0 con 0x7f042c15e6c0 2024-04-26T20:26:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0431200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 0x7f042c1f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5344001030 con 0x7f536814cd90 2024-04-26T20:26:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0433c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f042c142850 con 0x7f042c073480 2024-04-26T20:26:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0431200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 0x7f042c1f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55342/0 (socket says 172.21.15.43:55342) 2024-04-26T20:26:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0431200640 1 -- 172.21.15.43:0/13561826 learned_addr learned my addr 172.21.15.43:0/13561826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f0431c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042c15e6c0 0x7f042c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d841634a0 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7c00aca0 con 0x7f6d841634a0 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c012600 con 0x7f6d841634a0 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7f6d89400640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d84166330 con 0x7f6d841634a0 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7fb798a00640 1 -- 172.21.15.43:0/1225660911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb794073300 msgr2=0x7fb79406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.911+0000 7fb798a00640 1 --2- 172.21.15.43:0/1225660911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb794073300 0x7fb79406e3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f042be00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c0b2ec0 0x7f042c1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764042020 con 0x7f776c15ccc0 2024-04-26T20:26:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c1e5510 con 0x7f776c15ccc0 2024-04-26T20:26:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764038040 con 0x7f776c15ccc0 2024-04-26T20:26:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f6d89400640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841644d0 con 0x7f6d841634a0 2024-04-26T20:26:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f0431200640 1 -- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042c15e6c0 msgr2=0x7f042c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f0431200640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f042c15e6c0 0x7f042c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f0431200640 1 -- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c0b2ec0 msgr2=0x7f042c1f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7fb798a00640 1 -- 172.21.15.43:0/1225660911 shutdown_connections 2024-04-26T20:26:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7fb798a00640 1 --2- 172.21.15.43:0/1225660911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb794073300 0x7fb79406e3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7fb798a00640 1 -- 172.21.15.43:0/1225660911 >> 172.21.15.43:0/1225660911 conn(0x7fb7940b26b0 msgr2=0x7fb7940b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f0431200640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f042c0b2ec0 0x7f042c1f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f0431200640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c1f6c60 con 0x7f042c073480 2024-04-26T20:26:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1e5ad0 con 0x7f776c15ccc0 2024-04-26T20:26:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.912+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764042020 con 0x7f776c15ccc0 2024-04-26T20:26:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f535403ccb0 con 0x7f536814cd90 2024-04-26T20:26:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7fa86fe00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa85006c940 0x7fa85006ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0090b0 tx=0x7fa85c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d58001030 con 0x7f6d841634a0 2024-04-26T20:26:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7fb798a00640 1 -- 172.21.15.43:0/1225660911 shutdown_connections 2024-04-26T20:26:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7fb798a00640 1 -- 172.21.15.43:0/1225660911 wait complete. 2024-04-26T20:26:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f776c0b3f60 con 0x7f776c15ccc0 2024-04-26T20:26:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7f0431200640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 0x7f042c1f4f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f041c009a00 tx=0x7f041c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7fa874c00640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7fa8500760b0 0x7fa8500784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c041020 con 0x7f042c073480 2024-04-26T20:26:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.914+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c1f6f40 con 0x7f042c073480 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.913+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0b401d070 con 0x7ff0c4161640 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7ff0c0c00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff0a406c9c0 0x7ff0a406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f2733e00640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734073380 0x7f273415eda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f272c009e50 tx=0x7f272c02a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1560b71aabe1c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f2732000640 1 -- 172.21.15.43:0/991437659 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c036070 con 0x7f2734073380 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c06edf0 con 0x7f042c073480 2024-04-26T20:26:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c031600 con 0x7f042c073480 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c041020 con 0x7f042c073480 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f5364c00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f534806c970 0x7f534806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.915+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f53540ad570 con 0x7f536814cd90 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff0b408be30 con 0x7ff0c4161640 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7ff0c0c00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7ff0a4076130 0x7ff0a4078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0b4009e50 con 0x7ff0a4076130 2024-04-26T20:26:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff0b408c220 con 0x7ff0c4161640 2024-04-26T20:26:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f5364c00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f53480760f0 0x7f5348078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5354001d30 con 0x7f53480760f0 2024-04-26T20:26:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f53540ad920 con 0x7f536814cd90 2024-04-26T20:26:28.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f536ce00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f534806c970 0x7f534806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f2732000640 1 -- 172.21.15.43:0/991437659 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c031600 con 0x7f2734073380 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.916+0000 7f2732000640 1 -- 172.21.15.43:0/991437659 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c0319d0 con 0x7f2734073380 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 -- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734073380 msgr2=0x7f273415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734073380 0x7f273415eda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f272c009e50 tx=0x7f272c02a760 comp rx=0 tx=0).stop 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f536d800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f53480760f0 0x7f5348078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== osd.0 v2:172.21.15.43:6800/1201466725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa860001d30 con 0x7fa8500760b0 2024-04-26T20:26:28.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0404001030 con 0x7f042c073480 2024-04-26T20:26:28.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7fb798a00640 1 Processor -- start 2024-04-26T20:26:28.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 -- 172.21.15.43:0/991437659 shutdown_connections 2024-04-26T20:26:28.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734073380 0x7f273415eda0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d7c01a070 con 0x7f6d841634a0 2024-04-26T20:26:28.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 --2- 172.21.15.43:0/991437659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f273415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.917+0000 7f273a400640 1 -- 172.21.15.43:0/991437659 >> 172.21.15.43:0/991437659 conn(0x7f2734068940 msgr2=0x7f27340b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.918+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7ff0a4076130 0x7ff0a4078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.918+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f776403c910 con 0x7f776c15ccc0 2024-04-26T20:26:28.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7ff0c8e00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff0a406c9c0 0x7ff0a406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7f536ce00640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f534806c970 0x7f534806ee30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f535c006fd0 tx=0x7f535c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.918+0000 7fb798a00640 1 -- start start 2024-04-26T20:26:28.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7ff0c3e00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7ff0a4076130 0x7ff0a4078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7f273a400640 1 -- 172.21.15.43:0/991437659 shutdown_connections 2024-04-26T20:26:28.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7f6d7be00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6d6406c970 0x7f6d6406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7f273a400640 1 -- 172.21.15.43:0/991437659 wait complete. 2024-04-26T20:26:28.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d7c08c340 con 0x7f6d841634a0 2024-04-26T20:26:28.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f7763400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774c06ed00 0x7f774c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7fb798a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794073300 0x7fb7941553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.919+0000 7fb798a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 0x7fb794155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb798a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b3cc0 0x7fb794159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb798a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940b42a0 con 0x7fb794073300 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb798a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940b4120 con 0x7fb7940b3cc0 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb798a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb7940b4420 con 0x7fb7940b31d0 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb793e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b3cc0 0x7fb794159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb792a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 0x7fb794155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f77640acec0 con 0x7f776c15ccc0 2024-04-26T20:26:28.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f6d7be00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6d640760f0 0x7f6d64078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.921+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d7c00ae50 con 0x7f6d640760f0 2024-04-26T20:26:28.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.921+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d7c08c6f0 con 0x7f6d841634a0 2024-04-26T20:26:28.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f776b400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774c06ed00 0x7f774c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f536d800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f53480760f0 0x7f5348078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f7763400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f774c078380 0x7f774c07a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f041c044020 con 0x7f042c073480 2024-04-26T20:26:28.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7764050e50 con 0x7f774c078380 2024-04-26T20:26:28.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.921+0000 7f6d83e00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6d6406c970 0x7f6d6406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f77640ad350 con 0x7f776c15ccc0 2024-04-26T20:26:28.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb792a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 0x7fb794155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55352/0 (socket says 172.21.15.43:55352) 2024-04-26T20:26:28.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.920+0000 7fb792a00640 1 -- 172.21.15.43:0/1675007912 learned_addr learned my addr 172.21.15.43:0/1675007912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.921+0000 7ff0c8e00640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff0a406c9c0 0x7ff0a406ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc007380 tx=0x7ff0bc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.921+0000 7f776b400640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774c06ed00 0x7f774c0711c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77580074c0 tx=0x7f775800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb793400640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794073300 0x7fb7941553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7f6d83400640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6d640760f0 0x7f6d64078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb792a00640 1 -- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b3cc0 msgr2=0x7fb794159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7f6d83e00640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6d6406c970 0x7f6d6406ee30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d74001650 tx=0x7f6d74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb792a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b3cc0 0x7fb794159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb792a00640 1 -- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794073300 msgr2=0x7fb7941553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7f776aa00640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f774c078380 0x7f774c07a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7f273a400640 1 Processor -- start 2024-04-26T20:26:28.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb792a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794073300 0x7fb7941553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.922+0000 7fb792a00640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb794153860 con 0x7fb7940b31d0 2024-04-26T20:26:28.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7f273a400640 1 -- start start 2024-04-26T20:26:28.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== osd.2 v2:172.21.15.43:6824/637915066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0b4009e50 con 0x7ff0a4076130 2024-04-26T20:26:28.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== osd.4 v2:172.21.15.164:6808/2254882506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5354001d30 con 0x7f53480760f0 2024-04-26T20:26:28.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7fb792a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 0x7fb794155e30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c430 tx=0x7fb78400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7f776aa00640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f774c078380 0x7f774c07a7c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.923+0000 7f0428c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f040c06caf0 0x7f040c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784011790 con 0x7fb7940b31d0 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784017070 con 0x7fb7940b31d0 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f041c0ad000 con 0x7f042c073480 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f042be00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f040c06caf0 0x7f040c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f0428c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f040c076260 0x7f040c0786a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f040c078d70 con 0x7f040c076260 2024-04-26T20:26:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f041c0ad3f0 con 0x7f042c073480 2024-04-26T20:26:28.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734072f40 0x7f273416c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2734073380 0x7f2734143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f2734143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784011b70 con 0x7fb7940b31d0 2024-04-26T20:26:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f0431c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f040c076260 0x7f040c0786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2734142850 con 0x7f2734073380 2024-04-26T20:26:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27341426d0 con 0x7f2734072f40 2024-04-26T20:26:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f273a400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f273415e990 con 0x7f27340b2ec0 2024-04-26T20:26:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734072f40 0x7f273416c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.924+0000 7f6d83400640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6d640760f0 0x7f6d64078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f0431c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f040c076260 0x7f040c0786a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2733e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f2734143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2733e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f2734143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55358/0 (socket says 172.21.15.43:55358) 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2733e00640 1 -- 172.21.15.43:0/3406518525 learned_addr learned my addr 172.21.15.43:0/3406518525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2732a00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2734073380 0x7f2734143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7f2733400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734072f40 0x7f273416c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46460/0 (socket says 172.21.15.43:46460) 2024-04-26T20:26:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb794153b40 con 0x7fb7940b31d0 2024-04-26T20:26:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f042be00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f040c06caf0 0x7f040c06efb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f042c13e530 tx=0x7f0418009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.925+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb794156370 con 0x7fb7940b31d0 2024-04-26T20:26:28.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== osd.3 v2:172.21.15.43:6816/429853617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d7c00ae50 con 0x7f6d640760f0 2024-04-26T20:26:28.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb794153610 con 0x7fb7940b31d0 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f2732a00640 1 -- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 msgr2=0x7f2734143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f2732a00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f27340b2ec0 0x7f2734143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f2732a00640 1 -- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734072f40 msgr2=0x7f273416c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f2732a00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2734072f40 0x7f273416c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.926+0000 7f2732a00640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734159660 con 0x7f2734073380 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f2732a00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2734073380 0x7f2734143a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f272000c410 tx=0x7f272000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720017070 con 0x7f2734073380 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2720005230 con 0x7f2734073380 2024-04-26T20:26:28.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27200045e0 con 0x7f2734073380 2024-04-26T20:26:28.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== osd.5 v2:172.21.15.164:6800/2479038766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7764050e50 con 0x7f774c078380 2024-04-26T20:26:28.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.927+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27340cd800 con 0x7f2734073380 2024-04-26T20:26:28.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.928+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27340ce060 con 0x7f2734073380 2024-04-26T20:26:28.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.929+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== osd.1 v2:172.21.15.43:6808/299956676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f040c078d70 con 0x7f040c076260 2024-04-26T20:26:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.929+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb784011d10 con 0x7fb7940b31d0 2024-04-26T20:26:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.931+0000 7fb78b400640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77006c940 0x7fb77006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.931+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb78408c5f0 con 0x7fb7940b31d0 2024-04-26T20:26:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.931+0000 7fb78b400640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7fb7700760c0 0x7fb770078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.931+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb784014e70 con 0x7fb7700760c0 2024-04-26T20:26:28.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.931+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb78408c9a0 con 0x7fb7940b31d0 2024-04-26T20:26:28.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.932+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2734069350 con 0x7f2734073380 2024-04-26T20:26:28.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.932+0000 7fb793400640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77006c940 0x7fb77006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.933+0000 7fb793400640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77006c940 0x7fb77006ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005880 tx=0x7fb77c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.933+0000 7fb793e00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7fb7700760c0 0x7fb770078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.933+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f272001a070 con 0x7f2734073380 2024-04-26T20:26:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.934+0000 7fb793e00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7fb7700760c0 0x7fb770078500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.935+0000 7f272b400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f271406c8f0 0x7f271406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.935+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f272008b0d0 con 0x7f2734073380 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.936+0000 7f2733400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f271406c8f0 0x7f271406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.936+0000 7f272b400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7f2714076060 0x7f27140784a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.936+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2714078b70 con 0x7f2714076060 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.936+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2720092050 con 0x7f2734073380 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.936+0000 7f2733e00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7f2714076060 0x7f27140784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:28.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.937+0000 7f2733400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f271406c8f0 0x7f271406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f27240079a0 tx=0x7f2724008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.937+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== osd.7 v2:172.21.15.164:6820/3685734532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb784014e70 con 0x7fb7700760c0 2024-04-26T20:26:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.941+0000 7f2733e00640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7f2714076060 0x7f27140784a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.943+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== osd.6 v2:172.21.15.164:6816/3085699515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2714078b70 con 0x7f2714076060 2024-04-26T20:26:28.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.958+0000 7ff0ca200640 1 -- 172.21.15.43:0/1273316865 --> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0c40cfd10 con 0x7ff0a4076130 2024-04-26T20:26:28.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.959+0000 7ff0c0c00640 1 -- 172.21.15.43:0/1273316865 <== osd.2 v2:172.21.15.43:6824/637915066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff0c40cfd10 con 0x7ff0a4076130 2024-04-26T20:26:28.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.960+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 --> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa870140d60 con 0x7fa8500760b0 2024-04-26T20:26:28.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.960+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 --> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d58002dd0 con 0x7f6d640760f0 2024-04-26T20:26:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7f6d7be00640 1 -- 172.21.15.43:0/4201253410 <== osd.3 v2:172.21.15.43:6816/429853617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6d58002dd0 con 0x7f6d640760f0 2024-04-26T20:26:28.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7fa867e00640 1 -- 172.21.15.43:0/3389369987 <== osd.0 v2:172.21.15.43:6800/1201466725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa870140d60 con 0x7fa8500760b0 2024-04-26T20:26:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7ff0a4076130 msgr2=0x7ff0a4078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7ff0ba000640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7ff0a4076130 0x7ff0a4078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6d640760f0 msgr2=0x7f6d64078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.961+0000 7f6d79600640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6d640760f0 0x7f6d64078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff0a406c9c0 msgr2=0x7ff0a406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff0a406c9c0 0x7ff0a406ee80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc007380 tx=0x7ff0bc00b040 comp rx=0 tx=0).stop 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 msgr2=0x7ff0c4161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 --2- 172.21.15.43:0/1273316865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c4161640 0x7ff0c4161ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4163910 tx=0x7ff0b4007720 comp rx=0 tx=0).stop 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0c9800640 1 -- 172.21.15.43:0/1273316865 reap_dead start 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 shutdown_connections 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 >> 172.21.15.43:0/1273316865 conn(0x7ff0c4068090 msgr2=0x7ff0c4165e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6d6406c970 msgr2=0x7f6d6406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7f6d79600640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6d6406c970 0x7f6d6406ee30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d74001650 tx=0x7f6d74009040 comp rx=0 tx=0).stop 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 msgr2=0x7f6d841ed990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7f6d79600640 1 --2- 172.21.15.43:0/4201253410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d841634a0 0x7f6d841ed990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00cb40 tx=0x7f6d7c00a060 comp rx=0 tx=0).stop 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.962+0000 7f6d88a00640 1 -- 172.21.15.43:0/4201253410 reap_dead start 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 shutdown_connections 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 shutdown_connections 2024-04-26T20:26:28.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 >> 172.21.15.43:0/4201253410 conn(0x7f6d840b2640 msgr2=0x7f6d840b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 --> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f776c069350 con 0x7f774c078380 2024-04-26T20:26:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7ff0ba000640 1 -- 172.21.15.43:0/1273316865 wait complete. 2024-04-26T20:26:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7fa8500760b0 msgr2=0x7fa8500784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7fa875600640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7fa8500760b0 0x7fa8500784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 shutdown_connections 2024-04-26T20:26:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.963+0000 7f6d79600640 1 -- 172.21.15.43:0/4201253410 wait complete. 2024-04-26T20:26:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa85006c940 msgr2=0x7fa85006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa85006c940 0x7fa85006ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0090b0 tx=0x7fa85c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8700b4200 msgr2=0x7fa8701368e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 --2- 172.21.15.43:0/3389369987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8700b4200 0x7fa8701368e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa86000d5c0 tx=0x7fa86000da80 comp rx=0 tx=0).stop 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa874c00640 1 -- 172.21.15.43:0/3389369987 reap_dead start 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 shutdown_connections 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 >> 172.21.15.43:0/3389369987 conn(0x7fa8700b26b0 msgr2=0x7fa870169850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 shutdown_connections 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7fa875600640 1 -- 172.21.15.43:0/3389369987 wait complete. 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.964+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 --> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5344002dd0 con 0x7f53480760f0 2024-04-26T20:26:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.965+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 --> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0404002dd0 con 0x7f040c076260 2024-04-26T20:26:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.965+0000 7f7763400640 1 -- 172.21.15.43:0/79013580 <== osd.5 v2:172.21.15.164:6800/2479038766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f776c069350 con 0x7f774c078380 2024-04-26T20:26:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.965+0000 7f5364c00640 1 -- 172.21.15.43:0/1890516861 <== osd.4 v2:172.21.15.164:6808/2254882506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5344002dd0 con 0x7f53480760f0 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f53480760f0 msgr2=0x7f5348078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f774c078380 msgr2=0x7f774c07a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f774c078380 0x7f774c07a7c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774c06ed00 msgr2=0x7f774c0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774c06ed00 0x7f774c0711c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f77580074c0 tx=0x7f775800b040 comp rx=0 tx=0).stop 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 msgr2=0x7f776c153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 --2- 172.21.15.43:0/79013580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776c15ccc0 0x7f776c153500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f776402a700 tx=0x7f77640046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f53480760f0 0x7f5348078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f776be00640 1 -- 172.21.15.43:0/79013580 reap_dead start 2024-04-26T20:26:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f534806c970 msgr2=0x7f534806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f534806c970 0x7f534806ee30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f535c006fd0 tx=0x7f535c008040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 msgr2=0x7f53680b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f0428c00640 1 -- 172.21.15.43:0/13561826 <== osd.1 v2:172.21.15.43:6808/299956676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0404002dd0 con 0x7f040c076260 2024-04-26T20:26:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 shutdown_connections 2024-04-26T20:26:29.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 >> 172.21.15.43:0/79013580 conn(0x7f776c068090 msgr2=0x7f776c159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.967+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 shutdown_connections 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.967+0000 7f7771200640 1 -- 172.21.15.43:0/79013580 wait complete. 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.966+0000 7f536f800640 1 --2- 172.21.15.43:0/1890516861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f536814cd90 0x7f53680b1910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f535402a720 tx=0x7f53540046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.967+0000 7f536d800640 1 -- 172.21.15.43:0/1890516861 reap_dead start 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.967+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 shutdown_connections 2024-04-26T20:26:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.967+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 >> 172.21.15.43:0/1890516861 conn(0x7f53680b26b0 msgr2=0x7f536806f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.968+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f040c076260 msgr2=0x7f040c0786a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.968+0000 7f0433c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f040c076260 0x7f040c0786a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f040c06caf0 msgr2=0x7f040c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f0433c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f040c06caf0 0x7f040c06efb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f042c13e530 tx=0x7f0418009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 msgr2=0x7f042c1f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 --> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb794069350 con 0x7fb7700760c0 2024-04-26T20:26:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 shutdown_connections 2024-04-26T20:26:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f0433c00640 1 --2- 172.21.15.43:0/13561826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f042c073480 0x7f042c1f4f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f041c009a00 tx=0x7f041c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f536f800640 1 -- 172.21.15.43:0/1890516861 wait complete. 2024-04-26T20:26:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.969+0000 7f0431c00640 1 -- 172.21.15.43:0/13561826 reap_dead start 2024-04-26T20:26:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.970+0000 7fb78b400640 1 -- 172.21.15.43:0/1675007912 <== osd.7 v2:172.21.15.164:6820/3685734532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb794069350 con 0x7fb7700760c0 2024-04-26T20:26:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 shutdown_connections 2024-04-26T20:26:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7fb7700760c0 msgr2=0x7fb770078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7fb7700760c0 0x7fb770078500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 >> 172.21.15.43:0/13561826 conn(0x7f042c068940 msgr2=0x7f042c0b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77006c940 msgr2=0x7fb77006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77006c940 0x7fb77006ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005880 tx=0x7fb77c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 msgr2=0x7fb794155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 --2- 172.21.15.43:0/1675007912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b31d0 0x7fb794155e30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c430 tx=0x7fb78400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb793e00640 1 -- 172.21.15.43:0/1675007912 reap_dead start 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 shutdown_connections 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.971+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 >> 172.21.15.43:0/1675007912 conn(0x7fb7940b26b0 msgr2=0x7fb79415d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.972+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 shutdown_connections 2024-04-26T20:26:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.972+0000 7f0433c00640 1 -- 172.21.15.43:0/13561826 wait complete. 2024-04-26T20:26:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.973+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 shutdown_connections 2024-04-26T20:26:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.973+0000 7fb798a00640 1 -- 172.21.15.43:0/1675007912 wait complete. 2024-04-26T20:26:29.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.977+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 --> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27340b3c60 con 0x7f2714076060 2024-04-26T20:26:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.978+0000 7f272b400640 1 -- 172.21.15.43:0/3406518525 <== osd.6 v2:172.21.15.164:6816/3085699515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f27340b3c60 con 0x7f2714076060 2024-04-26T20:26:29.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.979+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7f2714076060 msgr2=0x7f27140784a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.979+0000 7f273a400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7f2714076060 0x7f27140784a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.980+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f271406c8f0 msgr2=0x7f271406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.980+0000 7f273a400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f271406c8f0 0x7f271406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f27240079a0 tx=0x7f2724008040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.981+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2734073380 msgr2=0x7f2734143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.981+0000 7f273a400640 1 --2- 172.21.15.43:0/3406518525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2734073380 0x7f2734143a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f272000c410 tx=0x7f272000c8d0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.982+0000 7f2733e00640 1 -- 172.21.15.43:0/3406518525 reap_dead start 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.982+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 shutdown_connections 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.982+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 >> 172.21.15.43:0/3406518525 conn(0x7f2734068940 msgr2=0x7f27340b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.984+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 shutdown_connections 2024-04-26T20:26:29.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:28.984+0000 7f273a400640 1 -- 172.21.15.43:0/3406518525 wait complete. 2024-04-26T20:26:29.011 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:26:29.012 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2024-04-26T20:26:29.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:26:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:26:29.014 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:26:29.015 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:26:29.016 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:26:29.017 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:29.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:26:29.018 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:26:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.339+0000 7f262e200640 1 Processor -- start 2024-04-26T20:26:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.340+0000 7f262e200640 1 -- start start 2024-04-26T20:26:29.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.345+0000 7fc31b600640 1 Processor -- start 2024-04-26T20:26:29.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.345+0000 7fc31b600640 1 -- start start 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f262e200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f26280b31d0 0x7f26280b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f262e200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2628072440 0x7f262806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f262e200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280b4dd0 con 0x7f26280b44c0 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f262e200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f26280b50b0 con 0x7f2628072440 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f262e200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f26280b5390 con 0x7f26280b31d0 2024-04-26T20:26:29.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f2626a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f26280b44c0 0x7f2628071d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41460/0 (socket says 172.21.15.43:41460) 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.342+0000 7f2626a00640 1 -- 172.21.15.43:0/2740328975 learned_addr learned my addr 172.21.15.43:0/2740328975 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.343+0000 7f2627e00640 1 --2- 172.21.15.43:0/2740328975 >> v2:172.21.15.43:3300/0 conn(0x7f2628072440 0x7f262806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.343+0000 7f2627400640 1 --2- 172.21.15.43:0/2740328975 >> v2:172.21.15.164:3300/0 conn(0x7f26280b31d0 0x7f26280b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.344+0000 7f2627e00640 1 -- 172.21.15.43:0/2740328975 >> v2:172.21.15.164:3300/0 conn(0x7f26280b31d0 msgr2=0x7f26280b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.344+0000 7f2627e00640 1 --2- 172.21.15.43:0/2740328975 >> v2:172.21.15.164:3300/0 conn(0x7f26280b31d0 0x7f26280b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.344+0000 7f2627e00640 1 -- 172.21.15.43:0/2740328975 >> v1:172.21.15.43:6789/0 conn(0x7f26280b44c0 legacy=0x7f2628071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.344+0000 7f2626000640 1 -- 172.21.15.43:0/2740328975 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1176908020 0 0) 0x7f26280b4dd0 con 0x7f26280b44c0 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.346+0000 7fb0b3200640 1 Processor -- start 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.346+0000 7fb0b3200640 1 -- start start 2024-04-26T20:26:29.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fb0b3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a404ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fb0b3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0a4053c30 0x7fb0a4007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fb0b3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a4057d80 con 0x7fb0a40589b0 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fb0b3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a4006250 con 0x7fb0a4053c30 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fb0b3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a4006530 con 0x7fb0a40e7700 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.344+0000 7f2627e00640 1 -- 172.21.15.43:0/2740328975 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26280b5670 con 0x7f2628072440 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.346+0000 7f2627e00640 1 --2- 172.21.15.43:0/2740328975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2628072440 0x7f262806dd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26280d08d0 tx=0x7f261802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a3d0b0e32a391 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.347+0000 7f2626000640 1 -- 172.21.15.43:0/2740328975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618037070 con 0x7f2628072440 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.347+0000 7f2626000640 1 -- 172.21.15.43:0/2740328975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2618032440 con 0x7f2628072440 2024-04-26T20:26:29.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.347+0000 7f2626000640 1 -- 172.21.15.43:0/2740328975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618037070 con 0x7f2628072440 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.345+0000 7f252a800640 1 Processor -- start 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.346+0000 7f252a800640 1 -- start start 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f252a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 0x7f25240d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f252a800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25240d3300 0x7f2524162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f252a800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25240d0d10 con 0x7f252415ef30 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f252a800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f25240d3a70 con 0x7f25240d3300 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f252a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f25240cfb20 con 0x7f252415f310 2024-04-26T20:26:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.347+0000 7fc31b600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc30c0e31b0 0x7fc30c006c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fc31b600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc30c0e7520 0x7fc30c057af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f262e200640 1 -- 172.21.15.43:0/2740328975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2628072440 msgr2=0x7f262806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f262e200640 1 --2- 172.21.15.43:0/2740328975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2628072440 0x7f262806dd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26280d08d0 tx=0x7f261802a270 comp rx=0 tx=0).stop 2024-04-26T20:26:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fc31b600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc30c0076e0 con 0x7fc30c007180 2024-04-26T20:26:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fc31b600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc30c057660 con 0x7fc30c0e7520 2024-04-26T20:26:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7fc31b600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc30c0079c0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f2522a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 0x7f25240d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f2522a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 0x7f25240d5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55370/0 (socket says 172.21.15.43:55370) 2024-04-26T20:26:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.348+0000 7f2522a00640 1 -- 172.21.15.43:0/42915446 learned_addr learned my addr 172.21.15.43:0/42915446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7f2523e00640 1 --2- 172.21.15.43:0/42915446 >> v2:172.21.15.164:3300/0 conn(0x7f25240d3300 0x7f2524162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7f262e200640 1 -- 172.21.15.43:0/2740328975 shutdown_connections 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7f262e200640 1 --2- 172.21.15.43:0/2740328975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2628072440 0x7f262806dd50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7f262e200640 1 --2- 172.21.15.43:0/2740328975 >> v2:172.21.15.164:3300/0 conn(0x7f26280b31d0 0x7f26280b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fb0b1200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0a4053c30 0x7fb0a4007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fb0b1200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0a4053c30 0x7fb0a4007010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46488/0 (socket says 172.21.15.43:46488) 2024-04-26T20:26:29.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7f262e200640 1 -- 172.21.15.43:0/2740328975 >> 172.21.15.43:0/2740328975 conn(0x7f26280b26b0 msgr2=0x7f26280b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fb0b1200640 1 -- 172.21.15.43:0/1562929282 learned_addr learned my addr 172.21.15.43:0/1562929282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fc313e00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc30c007180 0x7fc30c0e6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:44204/0 (socket says 172.21.15.43:44204) 2024-04-26T20:26:29.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fc313e00640 1 -- 172.21.15.43:0/1162182695 learned_addr learned my addr 172.21.15.43:0/1162182695 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:29.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fb0abe00640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a404ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.349+0000 7fc319600640 1 --2- 172.21.15.43:0/1162182695 >> v2:172.21.15.43:3300/0 conn(0x7fc30c0e7520 0x7fc30c057af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fb0b1200640 1 -- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 msgr2=0x7fb0a404ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fb0b1200640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a404ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fb0b1200640 1 -- 172.21.15.43:0/1562929282 >> v1:172.21.15.164:6789/0 conn(0x7fb0a40589b0 legacy=0x7fb0a4053520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fb0b1200640 1 -- 172.21.15.43:0/1562929282 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a4006810 con 0x7fb0a4053c30 2024-04-26T20:26:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fb0abe00640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a404ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fc318c00640 1 --2- 172.21.15.43:0/1162182695 >> v2:172.21.15.164:3300/0 conn(0x7fc30c0e31b0 0x7fc30c006c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fc313400640 1 -- 172.21.15.43:0/1162182695 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3879879445 0 0) 0x7fc30c0076e0 con 0x7fc30c007180 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fc313400640 1 -- 172.21.15.43:0/1162182695 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2fc003520 con 0x7fc30c007180 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fc319600640 1 -- 172.21.15.43:0/1162182695 >> v2:172.21.15.164:3300/0 conn(0x7fc30c0e31b0 msgr2=0x7fc30c006c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7fc319600640 1 --2- 172.21.15.43:0/1162182695 >> v2:172.21.15.164:3300/0 conn(0x7fc30c0e31b0 0x7fc30c006c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7fc319600640 1 -- 172.21.15.43:0/1162182695 >> v1:172.21.15.164:6789/0 conn(0x7fc30c007180 legacy=0x7fc30c0e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7fc319600640 1 -- 172.21.15.43:0/1162182695 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c0572f0 con 0x7fc30c0e7520 2024-04-26T20:26:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7f2523e00640 1 -- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 msgr2=0x7f25240d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7f2523e00640 1 --2- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 0x7f25240d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7f2523e00640 1 -- 172.21.15.43:0/42915446 >> v1:172.21.15.164:6789/0 conn(0x7f252415ef30 legacy=0x7f25240d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.350+0000 7f2523e00640 1 -- 172.21.15.43:0/42915446 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25240cfe00 con 0x7f25240d3300 2024-04-26T20:26:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7f262e200640 1 -- 172.21.15.43:0/2740328975 shutdown_connections 2024-04-26T20:26:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7f2523e00640 1 --2- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240d3300 0x7f2524162c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f250c0097a0 tx=0x7f250c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=527f0e055ce8394f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7fc319600640 1 --2- 172.21.15.43:0/1162182695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c057af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc31406b910 tx=0x7fc3140aee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872f39123ddd0b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7f2522000640 1 -- 172.21.15.43:0/42915446 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c037070 con 0x7f25240d3300 2024-04-26T20:26:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7f2522000640 1 -- 172.21.15.43:0/42915446 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c02ae50 con 0x7f25240d3300 2024-04-26T20:26:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7f2522000640 1 -- 172.21.15.43:0/42915446 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c032660 con 0x7f25240d3300 2024-04-26T20:26:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7fc313400640 1 -- 172.21.15.43:0/1162182695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140bc070 con 0x7fc30c0e7520 2024-04-26T20:26:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7fc313400640 1 -- 172.21.15.43:0/1162182695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3140afc80 con 0x7fc30c0e7520 2024-04-26T20:26:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.352+0000 7fc313400640 1 -- 172.21.15.43:0/1162182695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140b7640 con 0x7fc30c0e7520 2024-04-26T20:26:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.353+0000 7f252a800640 1 -- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240d3300 msgr2=0x7f2524162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.353+0000 7f252a800640 1 --2- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240d3300 0x7f2524162c00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f250c0097a0 tx=0x7f250c02a100 comp rx=0 tx=0).stop 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f252a800640 1 -- 172.21.15.43:0/42915446 shutdown_connections 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f252a800640 1 --2- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240d3300 0x7f2524162c00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f252a800640 1 --2- 172.21.15.43:0/42915446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415f310 0x7f25240d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f252a800640 1 -- 172.21.15.43:0/42915446 >> 172.21.15.43:0/42915446 conn(0x7f25240d4d20 msgr2=0x7f25240d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7fb0b1200640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a4053c30 0x7fb0a4007010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac051250 tx=0x7fb0ac0a65f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6bc85684ac5d257 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.351+0000 7f262e200640 1 -- 172.21.15.43:0/2740328975 wait complete. 2024-04-26T20:26:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f262e200640 1 Processor -- start 2024-04-26T20:26:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7fb0aaa00640 1 -- 172.21.15.43:0/1562929282 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac06e8a0 con 0x7fb0a4053c30 2024-04-26T20:26:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7fb0aaa00640 1 -- 172.21.15.43:0/1562929282 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac06ea40 con 0x7fb0a4053c30 2024-04-26T20:26:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7fb0aaa00640 1 -- 172.21.15.43:0/1562929282 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0ad400 con 0x7fb0a4053c30 2024-04-26T20:26:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.354+0000 7f262e200640 1 -- start start 2024-04-26T20:26:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7f252a800640 1 -- 172.21.15.43:0/42915446 shutdown_connections 2024-04-26T20:26:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7fc31b600640 1 -- 172.21.15.43:0/1162182695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 msgr2=0x7fc30c057af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7fc31b600640 1 --2- 172.21.15.43:0/1162182695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c057af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc31406b910 tx=0x7fc3140aee70 comp rx=0 tx=0).stop 2024-04-26T20:26:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7fb0b3200640 1 -- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a4053c30 msgr2=0x7fb0a4007010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7fb0b3200640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a4053c30 0x7fb0a4007010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac051250 tx=0x7fb0ac0a65f0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7fc31b600640 1 -- 172.21.15.43:0/1162182695 shutdown_connections 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fc31b600640 1 --2- 172.21.15.43:0/1162182695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c057af0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fc31b600640 1 --2- 172.21.15.43:0/1162182695 >> v2:172.21.15.164:3300/0 conn(0x7fc30c0e31b0 0x7fc30c006c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fc31b600640 1 -- 172.21.15.43:0/1162182695 >> 172.21.15.43:0/1162182695 conn(0x7fc30c03aa20 msgr2=0x7fc30c03ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fb0b3200640 1 -- 172.21.15.43:0/1562929282 shutdown_connections 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fb0b3200640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a4053c30 0x7fb0a4007010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fb0b3200640 1 --2- 172.21.15.43:0/1562929282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a404ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fb0b3200640 1 -- 172.21.15.43:0/1562929282 >> 172.21.15.43:0/1562929282 conn(0x7fb0a4058ed0 msgr2=0x7fb0a40592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fc31b600640 1 -- 172.21.15.43:0/1162182695 shutdown_connections 2024-04-26T20:26:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.355+0000 7f252a800640 1 -- 172.21.15.43:0/42915446 wait complete. 2024-04-26T20:26:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fb0b3200640 1 -- 172.21.15.43:0/1562929282 shutdown_connections 2024-04-26T20:26:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7fb0b3200640 1 -- 172.21.15.43:0/1562929282 wait complete. 2024-04-26T20:26:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f107e000640 1 Processor -- start 2024-04-26T20:26:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f107e000640 1 -- start start 2024-04-26T20:26:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f262e200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f262e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26280b31d0 0x7f26280b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f252a800640 1 Processor -- start 2024-04-26T20:26:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.357+0000 7f252a800640 1 -- start start 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f262e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280b44c0 0x7f26280b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f262e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26280cff00 con 0x7f26280b44c0 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f262e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26280cfd80 con 0x7f26280b31d0 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f262e200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f26280d0080 con 0x7f2628072440 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f2627400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f2627400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55384/0 (socket says 172.21.15.43:55384) 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f2627400640 1 -- 172.21.15.43:0/1414515194 learned_addr learned my addr 172.21.15.43:0/1414515194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.358+0000 7f2626a00640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26280b31d0 0x7f26280b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.356+0000 7fc31b600640 1 -- 172.21.15.43:0/1162182695 wait complete. 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7fc31b600640 1 Processor -- start 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7fc31b600640 1 -- start start 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f107e000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f10780b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f107e000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1078163930 con 0x7f107814cc60 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f107e000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780730c0 con 0x7f10780b3ef0 2024-04-26T20:26:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f107e000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1078073680 con 0x7f10780b3230 2024-04-26T20:26:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f252a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 0x7f25240b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f252a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 0x7f2524153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7f252a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 0x7f25241552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.359+0000 7fb0b3200640 1 Processor -- start 2024-04-26T20:26:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7fb0b3200640 1 -- start start 2024-04-26T20:26:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2627400640 1 -- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26280b31d0 msgr2=0x7f26280b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f107cc00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f107814cc60 0x7f10780728b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41476/0 (socket says 172.21.15.43:41476) 2024-04-26T20:26:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f107cc00640 1 -- 172.21.15.43:0/1146005907 learned_addr learned my addr 172.21.15.43:0/1146005907 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2627400640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26280b31d0 0x7f26280b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2627400640 1 -- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280b44c0 msgr2=0x7f26280b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2627400640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280b44c0 0x7f26280b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2522a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 0x7f2524153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886607935 0 0) 0x7f1078163930 con 0x7f107814cc60 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1060003520 con 0x7f107814cc60 2024-04-26T20:26:29.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077e00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f10780b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173992339 0 0) 0x7f1060003520 con 0x7f107814cc60 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 msgr2=0x7f10780b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f10780b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 >> v1:172.21.15.164:6789/0 conn(0x7f10780b3ef0 legacy=0x7f107806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780733a0 con 0x7f107814cc60 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f106c001670 con 0x7f107814cc60 2024-04-26T20:26:29.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f2627400640 1 -- 172.21.15.43:0/1414515194 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2628158ac0 con 0x7f2628072440 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2522a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 0x7f2524153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55392/0 (socket says 172.21.15.43:55392) 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2522a00640 1 -- 172.21.15.43:0/3964083724 learned_addr learned my addr 172.21.15.43:0/3964083724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.360+0000 7f2523e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 0x7f25241552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7fb0b3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7fb0b3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a40589b0 0x7fb0a404eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 >> v1:172.21.15.43:6789/0 conn(0x7f107814cc60 legacy=0x7f10780728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10600039e0 0x7f1060003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f1060004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1060004e10 0x7f1060005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1078073680 con 0x7f10600039e0 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f106000bcc0 con 0x7f1060004e10 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0b3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a403c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0b3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4057990 con 0x7fb0a4053c30 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0b3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a404cf60 con 0x7fb0a40589b0 2024-04-26T20:26:29.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0b3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a4057b10 con 0x7fb0a40e7700 2024-04-26T20:26:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f106000bec0 con 0x7f10780b3230 2024-04-26T20:26:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25240cf7f0 con 0x7f25240d3300 2024-04-26T20:26:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.361+0000 7f2523400640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 0x7f25240b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54700/0 (socket says 172.21.15.43:54700) 2024-04-26T20:26:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0abe00640 1 -- 172.21.15.43:0/3015979063 learned_addr learned my addr 172.21.15.43:0/3015979063 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7f2627400640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f262000efc0 tx=0x7f262000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0ab400640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a40589b0 0x7fb0a404eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fb0b1200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a403c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f25240d3770 con 0x7f25240b2fa0 2024-04-26T20:26:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2524163140 con 0x7f252415ef30 2024-04-26T20:26:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f107cc00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10600039e0 0x7f1060003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f1e1ac00640 1 Processor -- start 2024-04-26T20:26:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.362+0000 7fc31b600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 0x7fc30c0e10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f2523e00640 1 -- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 msgr2=0x7f2524153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f2523e00640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 0x7f2524153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f2523e00640 1 -- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 msgr2=0x7f25240b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f2523e00640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 0x7f25240b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f2523e00640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2524153a90 con 0x7f25240b2fa0 2024-04-26T20:26:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f1e1ac00640 1 -- start start 2024-04-26T20:26:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620018020 con 0x7f2628072440 2024-04-26T20:26:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2620014480 con 0x7f2628072440 2024-04-26T20:26:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620014830 con 0x7f2628072440 2024-04-26T20:26:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26282018a0 con 0x7f2628072440 2024-04-26T20:26:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f1077e00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f1060004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7f2523400640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 0x7f25240b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7f2523e00640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 0x7f25241552c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f252408d6d0 tx=0x7f250c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7fc31b600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 0x7fc30c03a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7fb0b1200640 1 -- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a40589b0 msgr2=0x7fb0a404eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7fb0b1200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a40589b0 0x7fb0a404eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7fb0b1200640 1 -- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 msgr2=0x7fb0a40e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7fb0b1200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7f107d600640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1060004e10 0x7f1060005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7f107cc00640 1 -- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 msgr2=0x7f1060004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.364+0000 7fb0b1200640 1 -- 172.21.15.43:0/3015979063 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a417df60 con 0x7fb0a40e7700 2024-04-26T20:26:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fb0abe00640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc31b600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c0da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fb0b1200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a403c8a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac06ab30 tx=0x7fb0ac06c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c041030 con 0x7f25240b2fa0 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc31b600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c03c230 con 0x7fc30c0e7520 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc31b600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c03c0b0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0bc070 con 0x7fb0a40e7700 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a40e7cf0 con 0x7fb0a40e7700 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a40e82b0 con 0x7fb0a40e7700 2024-04-26T20:26:29.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc31b600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc30c0e3590 con 0x7fc30c007180 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc318c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 0x7fc30c0e10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc318c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 0x7fc30c0e10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55428/0 (socket says 172.21.15.43:55428) 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc318c00640 1 -- 172.21.15.43:0/319789973 learned_addr learned my addr 172.21.15.43:0/319789973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7fc313e00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 0x7fc30c03a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250c02ac50 con 0x7f25240b2fa0 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c037040 con 0x7f25240b2fa0 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25240b19f0 con 0x7f25240b2fa0 2024-04-26T20:26:29.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2524155df0 con 0x7f25240b2fa0 2024-04-26T20:26:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7fc319600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c0da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.366+0000 7fda6aa00640 1 Processor -- start 2024-04-26T20:26:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0ade40 con 0x7fb0a40e7700 2024-04-26T20:26:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac0b7410 con 0x7fb0a40e7700 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.363+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2628201e60 con 0x7f2628072440 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7fda6aa00640 1 -- start start 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1e1ac00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 0x7f1e140b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1e1ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e1406f000 0x7f1e140d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1e1ac00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1414d290 con 0x7f1e1414ceb0 2024-04-26T20:26:29.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1e1ac00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1e140d1ab0 con 0x7f1e140cfe60 2024-04-26T20:26:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1e1ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1e140b5140 con 0x7f1e1406f000 2024-04-26T20:26:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f1e18c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e1406f000 0x7f1e140d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f1e18c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e1406f000 0x7f1e140d1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46560/0 (socket says 172.21.15.43:46560) 2024-04-26T20:26:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f1e18c00640 1 -- 172.21.15.43:0/4249156949 learned_addr learned my addr 172.21.15.43:0/4249156949 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2628069350 con 0x7f2628072440 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f107cc00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f1060004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f107cc00640 1 -- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1060004e10 msgr2=0x7f1060005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f107cc00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1060004e10 0x7f1060005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7fc313e00640 1 -- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 msgr2=0x7fc30c0e10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7fc313e00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 0x7fc30c0e10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7fc313e00640 1 -- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 msgr2=0x7fc30c0da9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f107cc00640 1 -- 172.21.15.43:0/1146005907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060003520 con 0x7f10600039e0 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f107d600640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1060004e10 0x7f1060005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.365+0000 7f1077e00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10780b3230 0x7f1060004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f107cc00640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10600039e0 0x7f1060003e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c0e0 tx=0x7f1060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09905cdda36d9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c007c20 con 0x7f10600039e0 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106c005ad0 con 0x7f10600039e0 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.367+0000 7f1e13400640 1 --2- 172.21.15.43:0/4249156949 >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 0x7f1e140b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7f1e18c00640 1 -- 172.21.15.43:0/4249156949 >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 msgr2=0x7f1e140b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.368+0000 7f1077400640 1 -- 172.21.15.43:0/1146005907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c007070 con 0x7f10600039e0 2024-04-26T20:26:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fc313e00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c0da9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f1e12a00640 1 -- 172.21.15.43:0/4249156949 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902263956 0 0) 0x7f1e1414d290 con 0x7f1e1414ceb0 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 0x7fda640d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6aa00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda640d0f10 con 0x7fda640d5850 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6aa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda640d0c30 con 0x7fda6415f170 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6aa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fda640b42a0 con 0x7fda640d3870 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f107e000640 1 -- 172.21.15.43:0/1146005907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10600039e0 msgr2=0x7f1060003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f107e000640 1 --2- 172.21.15.43:0/1146005907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10600039e0 0x7f1060003e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c0e0 tx=0x7f1060005830 comp rx=0 tx=0).stop 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f1e18c00640 1 --2- 172.21.15.43:0/4249156949 >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 0x7f1e140b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f1e18c00640 1 -- 172.21.15.43:0/4249156949 >> v1:172.21.15.164:6789/0 conn(0x7f1e1414ceb0 legacy=0x7f1e140cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7f1e18c00640 1 -- 172.21.15.43:0/4249156949 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e140b5420 con 0x7f1e1406f000 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6a000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda640d5850 0x7fda640b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41484/0 (socket says 172.21.15.43:41484) 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.369+0000 7fda6a000640 1 -- 172.21.15.43:0/3154508056 learned_addr learned my addr 172.21.15.43:0/3154508056 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f107d600640 1 -- 172.21.15.43:0/1146005907 reap_dead start 2024-04-26T20:26:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fb671a00640 1 Processor -- start 2024-04-26T20:26:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524069a60 con 0x7f25240b2fa0 2024-04-26T20:26:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fc313e00640 1 -- 172.21.15.43:0/319789973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c0e1790 con 0x7fc30c0e31b0 2024-04-26T20:26:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fb671a00640 1 -- start start 2024-04-26T20:26:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f107e000640 1 -- 172.21.15.43:0/1146005907 shutdown_connections 2024-04-26T20:26:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f107e000640 1 -- 172.21.15.43:0/1146005907 >> 172.21.15.43:0/1146005907 conn(0x7f10780b26b0 msgr2=0x7f10780b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fc319600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c0da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f107e000640 1 -- 172.21.15.43:0/1146005907 shutdown_connections 2024-04-26T20:26:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f1e13400640 1 --2- 172.21.15.43:0/4249156949 >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 0x7f1e140b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088005290 con 0x7fb0a40e7700 2024-04-26T20:26:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7fda68c00640 1 --2- 172.21.15.43:0/3154508056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 0x7fda640d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fda63e00640 1 -- 172.21.15.43:0/3154508056 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486629151 0 0) 0x7fda640d0c30 con 0x7fda6415f170 2024-04-26T20:26:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fda63e00640 1 -- 172.21.15.43:0/3154508056 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda4c003520 con 0x7fda6415f170 2024-04-26T20:26:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7f1e18c00640 1 --2- 172.21.15.43:0/4249156949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e140d1570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e140b3920 tx=0x7f1e0c02a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175d752293f4dd17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fc313e00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 0x7fc30c03a2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c430 tx=0x7fc30800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.370+0000 7f107e000640 1 -- 172.21.15.43:0/1146005907 wait complete. 2024-04-26T20:26:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fda68c00640 1 -- 172.21.15.43:0/3154508056 >> v1:172.21.15.164:6789/0 conn(0x7fda6415f170 legacy=0x7fda640d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fda68c00640 1 -- 172.21.15.43:0/3154508056 >> v1:172.21.15.43:6789/0 conn(0x7fda640d5850 legacy=0x7fda640b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7fda68c00640 1 -- 172.21.15.43:0/3154508056 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda640b4580 con 0x7fda640d3870 2024-04-26T20:26:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.371+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2620005d10 con 0x7f2628072440 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb671a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb66c072b70 0x7fb66c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb671a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66c146ba0 con 0x7fb66c0d1060 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb671a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66c0731c0 con 0x7fb66c1467c0 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb671a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb66c073dd0 con 0x7fb66c072b70 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb66b400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb66c0d1060 0x7fb66c06dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41494/0 (socket says 172.21.15.43:41494) 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.372+0000 7fb66b400640 1 -- 172.21.15.43:0/166417447 learned_addr learned my addr 172.21.15.43:0/166417447 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f250c044070 con 0x7f25240b2fa0 2024-04-26T20:26:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7fda68c00640 1 --2- 172.21.15.43:0/3154508056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 0x7fda640d2a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda500097d0 tx=0x7fda5002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ca6219d4b61b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7f107e000640 1 Processor -- start 2024-04-26T20:26:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7f107e000640 1 -- start start 2024-04-26T20:26:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0ac0b7670 con 0x7fb0a40e7700 2024-04-26T20:26:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.373+0000 7fb66a000640 1 --2- 172.21.15.43:0/166417447 >> v2:172.21.15.164:3300/0 conn(0x7fb66c072b70 0x7fb66c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1033128096 0 0) 0x7fb66c146ba0 con 0x7fb66c0d1060 2024-04-26T20:26:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb650003520 con 0x7fb66c0d1060 2024-04-26T20:26:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2278232831 0 0) 0x7fb650003520 con 0x7fb66c0d1060 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 >> v2:172.21.15.164:3300/0 conn(0x7fb66c072b70 msgr2=0x7fb66c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 --2- 172.21.15.43:0/166417447 >> v2:172.21.15.164:3300/0 conn(0x7fb66c072b70 0x7fb66c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 >> v1:172.21.15.164:6789/0 conn(0x7fb66c1467c0 legacy=0x7fb66c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c0734a0 con 0x7fb66c0d1060 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fb6640019f0 con 0x7fb66c0d1060 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 >> v1:172.21.15.43:6789/0 conn(0x7fb66c0d1060 legacy=0x7fb66c06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb650003a00 0x7fb650003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c073dd0 con 0x7fb650004800 2024-04-26T20:26:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10600039e0 0x7f107815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1078165410 0x7f10780ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f107814d040 con 0x7f107815f5f0 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f107806e0f0 con 0x7f1078165410 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f107e000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f10780b39b0 con 0x7f10600039e0 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f261f400640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f262008ca20 con 0x7f2628072440 2024-04-26T20:26:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7fb09fe00640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 0x7fb08c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb0ac1285e0 con 0x7fb0a40e7700 2024-04-26T20:26:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f251b400640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 0x7f250006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f250c02ae00 con 0x7f25240b2fa0 2024-04-26T20:26:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f107cc00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10600039e0 0x7f107815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f1077e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb65000c0c0 con 0x7fb650003a00 2024-04-26T20:26:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.374+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb65000c2c0 con 0x7fb6500051c0 2024-04-26T20:26:29.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f1077e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54738/0 (socket says 172.21.15.43:54738) 2024-04-26T20:26:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f1077e00640 1 -- 172.21.15.43:0/37949859 learned_addr learned my addr 172.21.15.43:0/37949859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7fb66aa00640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb650003a00 0x7fb650003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7fb66b400640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f107d600640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1078165410 0x7f10780ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f2626a00640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.375+0000 7f2523400640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 0x7f250006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 -- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10600039e0 msgr2=0x7f107815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10600039e0 0x7f107815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 -- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1078165410 msgr2=0x7f10780ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1078165410 0x7f10780ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 -- 172.21.15.43:0/37949859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780afce0 con 0x7f107815f5f0 2024-04-26T20:26:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb0abe00640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 0x7fb08c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7f1077e00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f106400d8f0 tx=0x7f106400ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 -- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6500051c0 msgr2=0x7fb65000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6500051c0 0x7fb65000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 -- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb650003a00 msgr2=0x7fb650003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb650003a00 0x7fb650003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66a000640 1 -- 172.21.15.43:0/166417447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb650003520 con 0x7fb650004800 2024-04-26T20:26:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.376+0000 7fb66b400640 1 -- 172.21.15.43:0/166417447 reap_dead start 2024-04-26T20:26:29.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.377+0000 7fb0abe00640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 0x7fb08c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0a403bca0 tx=0x7fb098008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.377+0000 7f2626a00640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 0x7f260806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2628154b80 tx=0x7f2614007b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.377+0000 7f2523400640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 0x7f250006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f251c001e00 tx=0x7f251c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064016020 con 0x7f107815f5f0 2024-04-26T20:26:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064004230 con 0x7f107815f5f0 2024-04-26T20:26:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1078163e30 con 0x7f107815f5f0 2024-04-26T20:26:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064008690 con 0x7f107815f5f0 2024-04-26T20:26:29.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781643f0 con 0x7f107815f5f0 2024-04-26T20:26:29.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.378+0000 7fb66a000640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb650004c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb654009aa0 tx=0x7fb650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a88e0c036c9f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.379+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654036070 con 0x7fb650004800 2024-04-26T20:26:29.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.379+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb654031400 con 0x7fb650004800 2024-04-26T20:26:29.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.379+0000 7fb669600640 1 -- 172.21.15.43:0/166417447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654009670 con 0x7fb650004800 2024-04-26T20:26:29.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.379+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1078069350 con 0x7f107815f5f0 2024-04-26T20:26:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 -- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 msgr2=0x7fb650004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb650004c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb654009aa0 tx=0x7fb650005830 comp rx=0 tx=0).stop 2024-04-26T20:26:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 -- 172.21.15.43:0/166417447 shutdown_connections 2024-04-26T20:26:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 --2- 172.21.15.43:0/166417447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb650004c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 -- 172.21.15.43:0/166417447 >> 172.21.15.43:0/166417447 conn(0x7fb66c0b2900 msgr2=0x7fb66c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 -- 172.21.15.43:0/166417447 shutdown_connections 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.380+0000 7fb671a00640 1 -- 172.21.15.43:0/166417447 wait complete. 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.381+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1064019020 con 0x7f107815f5f0 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.382+0000 7f1074c00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.382+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f106408c1a0 con 0x7f107815f5f0 2024-04-26T20:26:29.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.382+0000 7f107cc00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.382+0000 7fb671a00640 1 Processor -- start 2024-04-26T20:26:29.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.383+0000 7fb671a00640 1 -- start start 2024-04-26T20:26:29.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.383+0000 7f107cc00640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 0x7f105806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f106c032000 tx=0x7f106c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.383+0000 7fb671a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb671a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c1370f0 0x7fb66c143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb671a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb66c15c8d0 0x7fb66c132e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb671a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c0d0b20 con 0x7fb650004800 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb671a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c073a60 con 0x7fb66c1370f0 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb671a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb66c0d0ca0 con 0x7fb66c15c8d0 2024-04-26T20:26:29.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54754/0 (socket says 172.21.15.43:54754) 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 -- 172.21.15.43:0/1796242686 learned_addr learned my addr 172.21.15.43:0/1796242686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66b400640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb66c15c8d0 0x7fb66c132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 -- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb66c15c8d0 msgr2=0x7fb66c132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb66c15c8d0 0x7fb66c132e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 -- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c1370f0 msgr2=0x7fb66c143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c1370f0 0x7fb66c143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.384+0000 7fb66aa00640 1 -- 172.21.15.43:0/1796242686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c1e6580 con 0x7fb650004800 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.385+0000 7fb66aa00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00f590 tx=0x7fb65c00fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.386+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0052e0 con 0x7fb650004800 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.386+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c018070 con 0x7fb650004800 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.386+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c003830 con 0x7fb650004800 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.386+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66c1435d0 con 0x7fb650004800 2024-04-26T20:26:29.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.386+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66c1e62a0 con 0x7fb650004800 2024-04-26T20:26:29.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.387+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1064020020 con 0x7f107815f5f0 2024-04-26T20:26:29.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.387+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66c0d14b0 con 0x7fb650004800 2024-04-26T20:26:29.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.388+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb65c0039d0 con 0x7fb650004800 2024-04-26T20:26:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308017070 con 0x7fc30c0e31b0 2024-04-26T20:26:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc308005ce0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080072f0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c0e1a70 con 0x7fc30c0e31b0 2024-04-26T20:26:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7f1e12a00640 1 -- 172.21.15.43:0/4249156949 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c036070 con 0x7f1e1406f000 2024-04-26T20:26:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7f1e12a00640 1 -- 172.21.15.43:0/4249156949 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c031600 con 0x7f1e1406f000 2024-04-26T20:26:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7f1e12a00640 1 -- 172.21.15.43:0/4249156949 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c031a10 con 0x7f1e1406f000 2024-04-26T20:26:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.390+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c0e2030 con 0x7fc30c0e31b0 2024-04-26T20:26:29.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.389+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f250c07ac70 con 0x7f25240b2fa0 2024-04-26T20:26:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.390+0000 7fb662a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 0x7fb64806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.390+0000 7f1e1ac00640 1 -- 172.21.15.43:0/4249156949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 msgr2=0x7f1e140d1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.390+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/4249156949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e140d1570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e140b3920 tx=0x7f1e0c02a760 comp rx=0 tx=0).stop 2024-04-26T20:26:29.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.390+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb65c08bd40 con 0x7fb650004800 2024-04-26T20:26:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 -- 172.21.15.43:0/4249156949 shutdown_connections 2024-04-26T20:26:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/4249156949 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e140d1570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/4249156949 >> v2:172.21.15.43:3300/0 conn(0x7f1e140cfe60 0x7f1e140b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 -- 172.21.15.43:0/4249156949 >> 172.21.15.43:0/4249156949 conn(0x7f1e140b2900 msgr2=0x7f1e140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7fb66a000640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 0x7fb64806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 -- 172.21.15.43:0/4249156949 shutdown_connections 2024-04-26T20:26:29.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7f1e1ac00640 1 -- 172.21.15.43:0/4249156949 wait complete. 2024-04-26T20:26:29.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7fda63e00640 1 -- 172.21.15.43:0/3154508056 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50037070 con 0x7fda640d3870 2024-04-26T20:26:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.391+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30c0057d0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.392+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2620022070 con 0x7f2628072440 2024-04-26T20:26:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.392+0000 7fb66a000640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 0x7fb64806ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb654033000 tx=0x7fb65402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.393+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc30801a070 con 0x7fc30c0e31b0 2024-04-26T20:26:29.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.394+0000 7f1e1ac00640 1 Processor -- start 2024-04-26T20:26:29.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.394+0000 7f1e1ac00640 1 -- start start 2024-04-26T20:26:29.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.394+0000 7fc310c00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.394+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc30808b1b0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e141664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1414ceb0 0x7f1e14165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e140b5cc0 con 0x7f1e140cfe60 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e140b59e0 con 0x7f1e1406f000 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.395+0000 7f1e1ac00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1416d040 con 0x7f1e1414ceb0 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54760/0 (socket says 172.21.15.43:54760) 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 -- 172.21.15.43:0/2517558522 learned_addr learned my addr 172.21.15.43:0/2517558522 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13e00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e141664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e18c00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1414ceb0 0x7f1e14165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 -- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1414ceb0 msgr2=0x7f1e14165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1414ceb0 0x7f1e14165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 -- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 msgr2=0x7f1e141664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e141664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7f1e13400640 1 -- 172.21.15.43:0/2517558522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14166e60 con 0x7f1e140cfe60 2024-04-26T20:26:29.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.396+0000 7fc318c00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.397+0000 7f1e13e00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e141664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.397+0000 7f1e13400640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c7f0 tx=0x7f1e0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00015070 con 0x7f1e140cfe60 2024-04-26T20:26:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e00007a30 con 0x7f1e140cfe60 2024-04-26T20:26:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00007de0 con 0x7f1e140cfe60 2024-04-26T20:26:29.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7fc318c00640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 0x7fc2f406ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc300006fd0 tx=0x7fc300008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb66c0d14b0 con 0x7fb650004800 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e14166b80 con 0x7f1e140cfe60 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.398+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e14168b70 con 0x7f1e140cfe60 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.399+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1416ce60 con 0x7f1e140cfe60 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.402+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e00018070 con 0x7f1e140cfe60 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.403+0000 7f1e0be00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 0x7f1df406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.404+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1e0008be40 con 0x7f1e140cfe60 2024-04-26T20:26:29.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.404+0000 7f1e13e00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 0x7f1df406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.405+0000 7fda63e00640 1 -- 172.21.15.43:0/3154508056 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda50032440 con 0x7fda640d3870 2024-04-26T20:26:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.405+0000 7fda63e00640 1 -- 172.21.15.43:0/3154508056 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500327f0 con 0x7fda640d3870 2024-04-26T20:26:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.405+0000 7f1e13e00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 0x7f1df406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e141457c0 tx=0x7f1e04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.405+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0ac0c6020 con 0x7fb0a40e7700 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.406+0000 7fda6aa00640 1 -- 172.21.15.43:0/3154508056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 msgr2=0x7fda640d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.406+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3154508056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 0x7fda640d2a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fda500097d0 tx=0x7fda5002a210 comp rx=0 tx=0).stop 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.406+0000 7fda6aa00640 1 -- 172.21.15.43:0/3154508056 shutdown_connections 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.406+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3154508056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda640d3870 0x7fda640d2a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.406+0000 7fda6aa00640 1 -- 172.21.15.43:0/3154508056 >> 172.21.15.43:0/3154508056 conn(0x7fda640d4ca0 msgr2=0x7fda640d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.407+0000 7fda6aa00640 1 -- 172.21.15.43:0/3154508056 shutdown_connections 2024-04-26T20:26:29.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.408+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc308058c30 con 0x7fc30c0e31b0 2024-04-26T20:26:29.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.407+0000 7fda6aa00640 1 -- 172.21.15.43:0/3154508056 wait complete. 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.409+0000 7fda6aa00640 1 Processor -- start 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.410+0000 7fda6aa00640 1 -- start start 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda640d5850 0x7fda640b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 0x7fda640b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fda640d0850 con 0x7fda640d3870 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fda6415f550 con 0x7fda640d5850 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda6aa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fda640d09d0 con 0x7fda6415f170 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda69600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda69600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54772/0 (socket says 172.21.15.43:54772) 2024-04-26T20:26:29.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.411+0000 7fda69600640 1 -- 172.21.15.43:0/3193471366 learned_addr learned my addr 172.21.15.43:0/3193471366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.412+0000 7fda6a000640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 0x7fda640b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.412+0000 7fda68c00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda640d5850 0x7fda640b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.412+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e1416ce60 con 0x7f1e140cfe60 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.412+0000 7fda6a000640 1 -- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda640d5850 msgr2=0x7fda640b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda6a000640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda640d5850 0x7fda640b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda6a000640 1 -- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 msgr2=0x7fda641607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda6a000640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda6a000640 1 -- 172.21.15.43:0/3193471366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda64160e00 con 0x7fda6415f170 2024-04-26T20:26:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda69600640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.413+0000 7fda6a000640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 0x7fda640b7e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fda640b77c0 tx=0x7fda5c00cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.414+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c017070 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.414+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c00abd0 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.414+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c012400 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.414+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda641610e0 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.414+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda641616a0 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.416+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda640cf420 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.419+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda5c01a070 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.420+0000 7fda61600640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 0x7fda4406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.421+0000 7fda69600640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 0x7fda4406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.423+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fda5c08bfb0 con 0x7fda6415f170 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.423+0000 7fda69600640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 0x7fda4406ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda580011e0 tx=0x7fda58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:29.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.432+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda5c0bac00 con 0x7fda6415f170 2024-04-26T20:26:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.837+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f10780b0620 con 0x7f107815f5f0 2024-04-26T20:26:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.838+0000 7f1074c00640 1 -- 172.21.15.43:0/37949859 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f106405a320 con 0x7f107815f5f0 2024-04-26T20:26:29.840 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.840+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f262806f9f0 con 0x7f2628072440 2024-04-26T20:26:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.841+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f25240b60d0 con 0x7f25240b2fa0 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.841+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 msgr2=0x7f105806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.841+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 0x7f105806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f106c032000 tx=0x7f106c009030 comp rx=0 tx=0).stop 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.841+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 msgr2=0x7f1078164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.841+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f106400d8f0 tx=0x7f106400ddb0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f261f400640 1 -- 172.21.15.43:0/1414515194 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2620010040 con 0x7f2628072440 2024-04-26T20:26:29.843 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f251b400640 1 -- 172.21.15.43:0/3964083724 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f250c07a570 con 0x7f25240b2fa0 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 shutdown_connections 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1078165410 0x7f10780ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f107815f5f0 0x7f1078164ed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 --2- 172.21.15.43:0/37949859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10600039e0 0x7f107815f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.842+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 >> 172.21.15.43:0/37949859 conn(0x7f10780b26b0 msgr2=0x7f1078074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 shutdown_connections 2024-04-26T20:26:29.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 msgr2=0x7f250006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f107e000640 1 -- 172.21.15.43:0/37949859 wait complete. 2024-04-26T20:26:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 msgr2=0x7f260806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 0x7f260806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2628154b80 tx=0x7f2614007b80 comp rx=0 tx=0).stop 2024-04-26T20:26:29.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 msgr2=0x7f2628150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f262000efc0 tx=0x7f262000c490 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 shutdown_connections 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f260806c8f0 0x7f260806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280b44c0 0x7f26280b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26280b31d0 0x7f26280b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 --2- 172.21.15.43:0/1414515194 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2628072440 0x7f2628150d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 >> 172.21.15.43:0/1414515194 conn(0x7f26280b26b0 msgr2=0x7f26280b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 0x7f250006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f251c001e00 tx=0x7f251c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 msgr2=0x7f25241552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.845+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 0x7f25241552c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f252408d6d0 tx=0x7f250c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 shutdown_connections 2024-04-26T20:26:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f262e200640 1 -- 172.21.15.43:0/1414515194 wait complete. 2024-04-26T20:26:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 shutdown_connections 2024-04-26T20:26:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f250006c920 0x7f250006ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f252415ef30 0x7f2524153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25240b2fa0 0x7f25241552c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 --2- 172.21.15.43:0/3964083724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25240d3300 0x7f25240b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.846+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 >> 172.21.15.43:0/3964083724 conn(0x7f25240d4d20 msgr2=0x7f25240d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.847+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 shutdown_connections 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.847+0000 7f252a800640 1 -- 172.21.15.43:0/3964083724 wait complete. 2024-04-26T20:26:29.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.847+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb66c069350 con 0x7fb650004800 2024-04-26T20:26:29.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.849+0000 7fb662a00640 1 -- 172.21.15.43:0/1796242686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb66c1e6580 con 0x7fb650004800 2024-04-26T20:26:29.854 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.852+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb088005580 con 0x7fb0a40e7700 2024-04-26T20:26:29.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.853+0000 7fb09fe00640 1 -- 172.21.15.43:0/3015979063 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb0ac0f5f50 con 0x7fb0a40e7700 2024-04-26T20:26:29.856 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:29.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.857+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 msgr2=0x7fb64806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.857+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 0x7fb64806ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb654033000 tx=0x7fb65402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.857+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 msgr2=0x7fb66c15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.857+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00f590 tx=0x7fb65c00fa50 comp rx=0 tx=0).stop 2024-04-26T20:26:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 msgr2=0x7fb08c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 shutdown_connections 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb64806c920 0x7fb64806ede0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb66c15c8d0 0x7fb66c132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb66c1370f0 0x7fb66c143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 --2- 172.21.15.43:0/1796242686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb650004800 0x7fb66c15e970 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.858+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 >> 172.21.15.43:0/1796242686 conn(0x7fb66c0b2900 msgr2=0x7fb66c0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 0x7fb08c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0a403bca0 tx=0x7fb098008040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 msgr2=0x7fb0a403c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a403c8a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac06ab30 tx=0x7fb0ac06c0e0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 shutdown_connections 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb08c06c8f0 0x7fb08c06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0a40e7700 0x7fb0a403c8a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0a40589b0 0x7fb0a404eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 --2- 172.21.15.43:0/3015979063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0a4053c30 0x7fb0a40e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 >> 172.21.15.43:0/3015979063 conn(0x7fb0a4058ed0 msgr2=0x7fb0a4039da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 shutdown_connections 2024-04-26T20:26:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb671a00640 1 -- 172.21.15.43:0/1796242686 wait complete. 2024-04-26T20:26:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.859+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 shutdown_connections 2024-04-26T20:26:29.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.860+0000 7fb0b3200640 1 -- 172.21.15.43:0/3015979063 wait complete. 2024-04-26T20:26:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.860+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc30c0542d0 con 0x7fc30c0e31b0 2024-04-26T20:26:29.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.861+0000 7fc310c00640 1 -- 172.21.15.43:0/319789973 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc30805cf30 con 0x7fc30c0e31b0 2024-04-26T20:26:29.863 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.865+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 msgr2=0x7fc2f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.865+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 0x7fc2f406ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc300006fd0 tx=0x7fc300008040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.865+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 msgr2=0x7fc30c03a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.865+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 0x7fc30c03a2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c430 tx=0x7fc30800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 shutdown_connections 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc2f406c940 0x7fc2f406ee00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc30c0e7520 0x7fc30c0da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc30c0e31b0 0x7fc30c03a2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 --2- 172.21.15.43:0/319789973 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc30c007180 0x7fc30c0e10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 >> 172.21.15.43:0/319789973 conn(0x7fc30c03aa20 msgr2=0x7fc30c03dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 shutdown_connections 2024-04-26T20:26:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.866+0000 7fc31b600640 1 -- 172.21.15.43:0/319789973 wait complete. 2024-04-26T20:26:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.871+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1e140d1240 con 0x7f1e140cfe60 2024-04-26T20:26:29.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.872+0000 7f1e0be00640 1 -- 172.21.15.43:0/2517558522 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e14166e60 con 0x7f1e140cfe60 2024-04-26T20:26:29.876 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-26T20:26:29.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2024-04-26T20:26:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 msgr2=0x7f1df406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 0x7f1df406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e141457c0 tx=0x7f1e04009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 msgr2=0x7f1e1414a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c7f0 tx=0x7f1e0000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 shutdown_connections 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1df406c9c0 0x7f1df406ee80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e1414ceb0 0x7f1e14165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e140cfe60 0x7f1e1414a920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 --2- 172.21.15.43:0/2517558522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1406f000 0x7f1e141664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 >> 172.21.15.43:0/2517558522 conn(0x7f1e140b2900 msgr2=0x7f1e1406e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 shutdown_connections 2024-04-26T20:26:29.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.876+0000 7f1e1ac00640 1 -- 172.21.15.43:0/2517558522 wait complete. 2024-04-26T20:26:29.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2024-04-26T20:26:29.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-04-26T20:26:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.879+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fda6415ecd0 con 0x7fda6415f170 2024-04-26T20:26:29.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.880+0000 7fda61600640 1 -- 172.21.15.43:0/3193471366 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda6415ecd0 con 0x7fda6415f170 2024-04-26T20:26:29.882 INFO:teuthology.orchestra.run.smithi043.stdout:55834574857 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.883+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 msgr2=0x7fda4406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.883+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 0x7fda4406ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda580011e0 tx=0x7fda58009040 comp rx=0 tx=0).stop 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 msgr2=0x7fda640b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 0x7fda640b7e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fda640b77c0 tx=0x7fda5c00cf00 comp rx=0 tx=0).stop 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 shutdown_connections 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fda4406c940 0x7fda4406ee00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fda6415f170 0x7fda640b7e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda640d5850 0x7fda640b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 --2- 172.21.15.43:0/3193471366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda640d3870 0x7fda641607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:29.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 >> 172.21.15.43:0/3193471366 conn(0x7fda640d4ca0 msgr2=0x7fda640b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:29.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 shutdown_connections 2024-04-26T20:26:29.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:29.884+0000 7fda6aa00640 1 -- 172.21.15.43:0/3193471366 wait complete. 2024-04-26T20:26:29.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-04-26T20:26:29.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-04-26T20:26:29.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-26T20:26:29.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-04-26T20:26:29.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-26T20:26:30.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:26:30.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:26:30.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:26:30.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:26:30.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:26:30.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:26:30.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:26:30.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:26:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.146+0000 7f01b6200640 1 Processor -- start 2024-04-26T20:26:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.146+0000 7f01b6200640 1 -- start start 2024-04-26T20:26:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.148+0000 7f01b6200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01b00d1cf0 0x7f01b00d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.148+0000 7f01b6200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b4020 0x7f01b00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01b6200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b00c85e0 con 0x7f01b00b3c40 2024-04-26T20:26:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01b6200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f01b00d03d0 con 0x7f01b00d1cf0 2024-04-26T20:26:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01b6200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01b00d06b0 con 0x7f01b00b4020 2024-04-26T20:26:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01afe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01b00d1cf0 0x7f01b00d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01afe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01b00d1cf0 0x7f01b00d57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46616/0 (socket says 172.21.15.43:46616) 2024-04-26T20:26:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.149+0000 7f01afe00640 1 -- 172.21.15.43:0/1216996783 learned_addr learned my addr 172.21.15.43:0/1216996783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.150+0000 7f01b5800640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b4020 0x7f01b00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.150+0000 7f01af400640 1 -- 172.21.15.43:0/1216996783 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166462234 0 0) 0x7f01b00c85e0 con 0x7f01b00b3c40 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.151+0000 7f01af400640 1 -- 172.21.15.43:0/1216996783 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0194003520 con 0x7f01b00b3c40 2024-04-26T20:26:31.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.151+0000 7f01afe00640 1 -- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b4020 msgr2=0x7f01b00b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.151+0000 7f01afe00640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b4020 0x7f01b00b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.151+0000 7f01afe00640 1 -- 172.21.15.43:0/1216996783 >> v1:172.21.15.43:6789/0 conn(0x7f01b00b3c40 legacy=0x7f01b00d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.151+0000 7f01afe00640 1 -- 172.21.15.43:0/1216996783 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b00d0990 con 0x7f01b00d1cf0 2024-04-26T20:26:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.153+0000 7f01afe00640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00d1cf0 0x7f01b00d57a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098a0 tx=0x7f01a002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3acd9a32358bbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.153+0000 7f01af400640 1 -- 172.21.15.43:0/1216996783 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037070 con 0x7f01b00d1cf0 2024-04-26T20:26:31.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.153+0000 7f01af400640 1 -- 172.21.15.43:0/1216996783 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0032440 con 0x7f01b00d1cf0 2024-04-26T20:26:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.153+0000 7f01af400640 1 -- 172.21.15.43:0/1216996783 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0032810 con 0x7f01b00d1cf0 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.154+0000 7f01b6200640 1 -- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00d1cf0 msgr2=0x7f01b00d57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.154+0000 7f01b6200640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00d1cf0 0x7f01b00d57a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098a0 tx=0x7f01a002a270 comp rx=0 tx=0).stop 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 -- 172.21.15.43:0/1216996783 shutdown_connections 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b4020 0x7f01b00b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 --2- 172.21.15.43:0/1216996783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00d1cf0 0x7f01b00d57a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 -- 172.21.15.43:0/1216996783 >> 172.21.15.43:0/1216996783 conn(0x7f01b00d4c80 msgr2=0x7f01b00d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 -- 172.21.15.43:0/1216996783 shutdown_connections 2024-04-26T20:26:31.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.155+0000 7f01b6200640 1 -- 172.21.15.43:0/1216996783 wait complete. 2024-04-26T20:26:31.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.158+0000 7f01b6200640 1 Processor -- start 2024-04-26T20:26:31.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.159+0000 7f01b6200640 1 -- start start 2024-04-26T20:26:31.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.161+0000 7f01b6200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.161+0000 7f01b6200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.162+0000 7f01b6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.162+0000 7f01b4e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.162+0000 7f01b4e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55510/0 (socket says 172.21.15.43:55510) 2024-04-26T20:26:31.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.162+0000 7f01afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.163+0000 7f01afe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46628/0 (socket says 172.21.15.43:46628) 2024-04-26T20:26:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.163+0000 7f01b5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.164+0000 7f01b5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54776/0 (socket says 172.21.15.43:54776) 2024-04-26T20:26:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.164+0000 7f01b4e00640 1 -- 172.21.15.43:0/3807447094 learned_addr learned my addr 172.21.15.43:0/3807447094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.164+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01b00c7ec0 con 0x7f01b00d1cf0 2024-04-26T20:26:31.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.164+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01b00c7d40 con 0x7f01b00b4020 2024-04-26T20:26:31.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.164+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01b00c8040 con 0x7f01b00b3c40 2024-04-26T20:26:31.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.165+0000 7f01b5800640 1 -- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 msgr2=0x7f01b00b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.165+0000 7f01b5800640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.165+0000 7f01b5800640 1 -- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 msgr2=0x7f01b01653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.165+0000 7f01b5800640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.165+0000 7f01b5800640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b00bab50 con 0x7f01b00d1cf0 2024-04-26T20:26:31.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.166+0000 7f01b5800640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01a800efa0 tx=0x7f01a800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.167+0000 7f01afe00640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:31.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.167+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8018020 con 0x7f01b00d1cf0 2024-04-26T20:26:31.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.168+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8014480 con 0x7f01b00d1cf0 2024-04-26T20:26:31.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.168+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8014850 con 0x7f01b00d1cf0 2024-04-26T20:26:31.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.168+0000 7f01b4e00640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:31.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.168+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b00ba590 con 0x7f01b00d1cf0 2024-04-26T20:26:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.172+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b00b8d00 con 0x7f01b00d1cf0 2024-04-26T20:26:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.173+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01a801b020 con 0x7f01b00d1cf0 2024-04-26T20:26:31.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.175+0000 7f01acc00640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 0x7f019006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.176+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f01a808bde0 con 0x7f01b00d1cf0 2024-04-26T20:26:31.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.177+0000 7f01b4e00640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 0x7f019006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.178+0000 7f01b4e00640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 0x7f019006ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f019c0011e0 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.179+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b0069350 con 0x7f01b00d1cf0 2024-04-26T20:26:31.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.183+0000 7fdf5f000640 1 Processor -- start 2024-04-26T20:26:31.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.183+0000 7fdf5f000640 1 -- start start 2024-04-26T20:26:31.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.186+0000 7fdf5f000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf5003c0a0 0x7fdf500e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.187+0000 7fdf5f000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 0x7fdf50054100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.187+0000 7fdf5f000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf50058810 con 0x7fdf500e3160 2024-04-26T20:26:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.188+0000 7fdf5d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf5003c0a0 0x7fdf500e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.188+0000 7fdf5d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf5003c0a0 0x7fdf500e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46634/0 (socket says 172.21.15.43:46634) 2024-04-26T20:26:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.188+0000 7fdf5d200640 1 -- 172.21.15.43:0/451877658 learned_addr learned my addr 172.21.15.43:0/451877658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.188+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdf50007b70 con 0x7fdf50053cb0 2024-04-26T20:26:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.188+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdf50058af0 con 0x7fdf5003c0a0 2024-04-26T20:26:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.189+0000 7fdf5e600640 1 --2- 172.21.15.43:0/451877658 >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 0x7fdf50054100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.189+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01a8010040 con 0x7f01b00d1cf0 2024-04-26T20:26:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.190+0000 7fdf57e00640 1 -- 172.21.15.43:0/451877658 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937299030 0 0) 0x7fdf50058810 con 0x7fdf500e3160 2024-04-26T20:26:31.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.190+0000 7fdf5d200640 1 -- 172.21.15.43:0/451877658 >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 msgr2=0x7fdf50054100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.190+0000 7fdf5d200640 1 --2- 172.21.15.43:0/451877658 >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 0x7fdf50054100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.190+0000 7fdf5d200640 1 -- 172.21.15.43:0/451877658 >> v1:172.21.15.164:6789/0 conn(0x7fdf500e3160 legacy=0x7fdf50058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.190+0000 7fdf5d200640 1 -- 172.21.15.43:0/451877658 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf50006360 con 0x7fdf5003c0a0 2024-04-26T20:26:31.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.192+0000 7fdf5e600640 1 --2- 172.21.15.43:0/451877658 >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 0x7fdf50054100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.192+0000 7fdf5d200640 1 --2- 172.21.15.43:0/451877658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5003c0a0 0x7fdf500e7780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf58051250 tx=0x7fdf580a66a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5136bce87cd8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.193+0000 7fdf57e00640 1 -- 172.21.15.43:0/451877658 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5806e8a0 con 0x7fdf5003c0a0 2024-04-26T20:26:31.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.193+0000 7fdf57e00640 1 -- 172.21.15.43:0/451877658 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5806ea40 con 0x7fdf5003c0a0 2024-04-26T20:26:31.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.194+0000 7fdf57e00640 1 -- 172.21.15.43:0/451877658 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580ad400 con 0x7fdf5003c0a0 2024-04-26T20:26:31.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5003c0a0 msgr2=0x7fdf500e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 --2- 172.21.15.43:0/451877658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5003c0a0 0x7fdf500e7780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf58051250 tx=0x7fdf580a66a0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 shutdown_connections 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 --2- 172.21.15.43:0/451877658 >> v2:172.21.15.43:3300/0 conn(0x7fdf50053cb0 0x7fdf50054100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 --2- 172.21.15.43:0/451877658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf5003c0a0 0x7fdf500e7780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.195+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 >> 172.21.15.43:0/451877658 conn(0x7fdf50058ee0 msgr2=0x7fdf500592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.196+0000 7f5535a00640 1 Processor -- start 2024-04-26T20:26:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.196+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 shutdown_connections 2024-04-26T20:26:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.196+0000 7fdf5f000640 1 -- 172.21.15.43:0/451877658 wait complete. 2024-04-26T20:26:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.196+0000 7f5535a00640 1 -- start start 2024-04-26T20:26:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.199+0000 7f5535a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55280e7a50 0x7f5528049b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.199+0000 7f5535a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 0x7f55280548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.199+0000 7f5535a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552803c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.199+0000 7f5535a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f55280e7e30 con 0x7f55280e7a50 2024-04-26T20:26:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f5535a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f552803b5a0 con 0x7f552804a0a0 2024-04-26T20:26:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f5535a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f552803b880 con 0x7f5528054de0 2024-04-26T20:26:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f552f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 0x7f55280548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f552f400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 0x7f55280548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46642/0 (socket says 172.21.15.43:46642) 2024-04-26T20:26:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f552f400640 1 -- 172.21.15.43:0/2008100189 learned_addr learned my addr 172.21.15.43:0/2008100189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.200+0000 7f5535000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552803c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f552fe00640 1 --2- 172.21.15.43:0/2008100189 >> v2:172.21.15.43:3300/0 conn(0x7f55280e7a50 0x7f5528049b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7fdf5f000640 1 Processor -- start 2024-04-26T20:26:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.202+0000 7fdf5f000640 1 -- start start 2024-04-26T20:26:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f5535000640 1 -- 172.21.15.43:0/2008100189 >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 msgr2=0x7f55280548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f5535000640 1 --2- 172.21.15.43:0/2008100189 >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 0x7f55280548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f5535000640 1 -- 172.21.15.43:0/2008100189 >> v2:172.21.15.43:3300/0 conn(0x7f55280e7a50 msgr2=0x7f5528049b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f5535000640 1 --2- 172.21.15.43:0/2008100189 >> v2:172.21.15.43:3300/0 conn(0x7f55280e7a50 0x7f5528049b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.201+0000 7f5535000640 1 -- 172.21.15.43:0/2008100189 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552803bb60 con 0x7f5528054de0 2024-04-26T20:26:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.203+0000 7f5535000640 1 --2- 172.21.15.43:0/2008100189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552803c360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f553006b910 tx=0x7f55300a6590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471d27607997e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.203+0000 7fdf5f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.203+0000 7fdf5f000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf50053cb0 0x7fdf500e4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.203+0000 7f552ea00640 1 -- 172.21.15.43:0/2008100189 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553006e8a0 con 0x7f5528054de0 2024-04-26T20:26:31.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7f552ea00640 1 -- 172.21.15.43:0/2008100189 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553006ea40 con 0x7f5528054de0 2024-04-26T20:26:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7fdf5f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 0x7fdf500390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7fdf5f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf500e3540 con 0x7fdf5003c0a0 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7fdf5f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50054640 con 0x7fdf500e3160 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7fdf5f000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf50006b60 con 0x7fdf50053cb0 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.204+0000 7fdf5dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.205+0000 7fdf5d200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf50053cb0 0x7fdf500e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.205+0000 7fdf5dc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54796/0 (socket says 172.21.15.43:54796) 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.205+0000 7fdf5dc00640 1 -- 172.21.15.43:0/3409775458 learned_addr learned my addr 172.21.15.43:0/3409775458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.205+0000 7fdf5e600640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 0x7fdf500390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.205+0000 7f5535a00640 1 -- 172.21.15.43:0/2008100189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 msgr2=0x7f552803c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7f552ea00640 1 -- 172.21.15.43:0/2008100189 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553006e8a0 con 0x7f5528054de0 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5e600640 1 -- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf50053cb0 msgr2=0x7fdf500e4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5e600640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf50053cb0 0x7fdf500e4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5e600640 1 -- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 msgr2=0x7fdf5004f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5e600640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5e600640 1 -- 172.21.15.43:0/3409775458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf500da800 con 0x7fdf500e3160 2024-04-26T20:26:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7fdf5dc00640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.206+0000 7f5535a00640 1 --2- 172.21.15.43:0/2008100189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552803c360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f553006b910 tx=0x7f55300a6590 comp rx=0 tx=0).stop 2024-04-26T20:26:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 -- 172.21.15.43:0/2008100189 shutdown_connections 2024-04-26T20:26:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 --2- 172.21.15.43:0/2008100189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552803c360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7fdf5e600640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 0x7fdf500390f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e880 tx=0x7fdf4400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 --2- 172.21.15.43:0/2008100189 >> v2:172.21.15.164:3300/0 conn(0x7f552804a0a0 0x7f55280548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 --2- 172.21.15.43:0/2008100189 >> v2:172.21.15.43:3300/0 conn(0x7f55280e7a50 0x7f5528049b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 -- 172.21.15.43:0/2008100189 >> 172.21.15.43:0/2008100189 conn(0x7f5528056cd0 msgr2=0x7f55280570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 -- 172.21.15.43:0/2008100189 shutdown_connections 2024-04-26T20:26:31.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7f5535a00640 1 -- 172.21.15.43:0/2008100189 wait complete. 2024-04-26T20:26:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.207+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44017070 con 0x7fdf500e3160 2024-04-26T20:26:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.208+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44004020 con 0x7fdf500e3160 2024-04-26T20:26:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.208+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf440043f0 con 0x7fdf500e3160 2024-04-26T20:26:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.209+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf50180c10 con 0x7fdf500e3160 2024-04-26T20:26:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.209+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf50181160 con 0x7fdf500e3160 2024-04-26T20:26:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.210+0000 7f5535a00640 1 Processor -- start 2024-04-26T20:26:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.210+0000 7f5535a00640 1 -- start start 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552804afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55280e7a50 0x7f552804b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5528039cb0 con 0x7f552804a0a0 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f552804a5b0 con 0x7f55280e7a50 2024-04-26T20:26:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f5535a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5528039e30 con 0x7f5528054de0 2024-04-26T20:26:31.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f552fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f552fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54802/0 (socket says 172.21.15.43:54802) 2024-04-26T20:26:31.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7f552fe00640 1 -- 172.21.15.43:0/1569332977 learned_addr learned my addr 172.21.15.43:0/1569332977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 -- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 msgr2=0x7f552804afb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:26:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552804afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 -- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55280e7a50 msgr2=0x7f552804b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:26:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55280e7a50 0x7f552804b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 -- 172.21.15.43:0/1569332977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5528008da0 con 0x7f552804a0a0 2024-04-26T20:26:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.212+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf50005390 con 0x7fdf500e3160 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.213+0000 7f552fe00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f551c00e9b0 tx=0x7f551c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.214+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c017070 con 0x7f552804a0a0 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.214+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551c00c9a0 con 0x7f552804a0a0 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.214+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c00cd90 con 0x7f552804a0a0 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.214+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5528009080 con 0x7f552804a0a0 2024-04-26T20:26:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.214+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5528009640 con 0x7f552804a0a0 2024-04-26T20:26:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.216+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55280057e0 con 0x7f552804a0a0 2024-04-26T20:26:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.217+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf4400c940 con 0x7fdf500e3160 2024-04-26T20:26:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.218+0000 7fdf55600640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 0x7fdf3806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.219+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdf4408bc10 con 0x7fdf500e3160 2024-04-26T20:26:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.218+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f551c01a070 con 0x7f552804a0a0 2024-04-26T20:26:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.219+0000 7fdf5dc00640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 0x7fdf3806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.220+0000 7f5523e00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 0x7f551006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.220+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f551c08c200 con 0x7f552804a0a0 2024-04-26T20:26:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.221+0000 7f552f400640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 0x7f551006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.223+0000 7f552f400640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 0x7f551006ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f552804a870 tx=0x7f5524009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.223+0000 7fdf5dc00640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 0x7fdf3806ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007fd0 tx=0x7fdf4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.226+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f551c05a240 con 0x7f552804a0a0 2024-04-26T20:26:31.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.228+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf44059690 con 0x7fdf500e3160 2024-04-26T20:26:31.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.266+0000 7f6b8f200640 1 Processor -- start 2024-04-26T20:26:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.266+0000 7f6b8f200640 1 -- start start 2024-04-26T20:26:31.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.268+0000 7f96fe000640 1 Processor -- start 2024-04-26T20:26:31.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.269+0000 7f6b8f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.269+0000 7f6b8f200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9006ea10 con 0x7f6b90141540 2024-04-26T20:26:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.270+0000 7f6b8f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9006ecf0 con 0x7f6b900b43d0 2024-04-26T20:26:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.270+0000 7f6b8e800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.270+0000 7f6b8e800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55522/0 (socket says 172.21.15.43:55522) 2024-04-26T20:26:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.270+0000 7f6b8e800640 1 -- 172.21.15.43:0/3589429062 learned_addr learned my addr 172.21.15.43:0/3589429062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.270+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6b90141920 con 0x7f6b90073300 2024-04-26T20:26:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.269+0000 7f96fe000640 1 -- start start 2024-04-26T20:26:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.271+0000 7f6b8ca00640 1 -- 172.21.15.43:0/3589429062 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759007054 0 0) 0x7f6b9006ea10 con 0x7f6b90141540 2024-04-26T20:26:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.271+0000 7f6b8ca00640 1 -- 172.21.15.43:0/3589429062 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b74003520 con 0x7f6b90141540 2024-04-26T20:26:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.271+0000 7f96f7400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f96f00e7810 0x7f96f00eb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41532/0 (socket says 172.21.15.43:41532) 2024-04-26T20:26:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.272+0000 7f96f7400640 1 -- 172.21.15.43:0/2719079721 learned_addr learned my addr 172.21.15.43:0/2719079721 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.272+0000 7f96fe000640 1 --2- 172.21.15.43:0/2719079721 >> v2:172.21.15.164:3300/0 conn(0x7f96f0038270 0x7f96f00eba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.273+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f00e7bf0 con 0x7f96f00e7810 2024-04-26T20:26:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.272+0000 7f6b8e800640 1 -- 172.21.15.43:0/3589429062 >> v1:172.21.15.164:6789/0 conn(0x7f6b900b43d0 legacy=0x7f6b9006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.272+0000 7f6b8e800640 1 -- 172.21.15.43:0/3589429062 >> v1:172.21.15.43:6789/0 conn(0x7f6b90141540 legacy=0x7f6b900b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.272+0000 7f6b8e800640 1 -- 172.21.15.43:0/3589429062 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9006efd0 con 0x7f6b90073300 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.273+0000 7f96f6000640 1 -- 172.21.15.43:0/2719079721 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347616098 0 0) 0x7f96f00e7bf0 con 0x7f96f00e7810 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.273+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f0057f10 con 0x7f96f0006190 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.274+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f96f00ebf40 con 0x7f96f0038270 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.274+0000 7f96f6a00640 1 --2- 172.21.15.43:0/2719079721 >> v2:172.21.15.164:3300/0 conn(0x7f96f0038270 0x7f96f00eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.274+0000 7f6b8e800640 1 --2- 172.21.15.43:0/3589429062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b880098a0 tx=0x7f6b8802a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0699c6f6997bd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.274+0000 7f6b8ca00640 1 -- 172.21.15.43:0/3589429062 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88037070 con 0x7f6b90073300 2024-04-26T20:26:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.274+0000 7f96f6a00640 1 -- 172.21.15.43:0/2719079721 >> v1:172.21.15.164:6789/0 conn(0x7f96f0006190 legacy=0x7f96f0006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.275+0000 7f6b8ca00640 1 -- 172.21.15.43:0/3589429062 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b88032440 con 0x7f6b90073300 2024-04-26T20:26:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.275+0000 7f96f6a00640 1 -- 172.21.15.43:0/2719079721 >> v1:172.21.15.43:6789/0 conn(0x7f96f00e7810 legacy=0x7f96f00eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.275+0000 7f96f6a00640 1 -- 172.21.15.43:0/2719079721 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f00581f0 con 0x7f96f0038270 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.275+0000 7f6b8ca00640 1 -- 172.21.15.43:0/3589429062 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880327f0 con 0x7f6b90073300 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 msgr2=0x7f6b9006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3589429062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6b880098a0 tx=0x7f6b8802a2c0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 shutdown_connections 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3589429062 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90073300 0x7f6b9006e3f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 >> 172.21.15.43:0/3589429062 conn(0x7f6b900b26b0 msgr2=0x7f6b900b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 shutdown_connections 2024-04-26T20:26:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.276+0000 7f6b8f200640 1 -- 172.21.15.43:0/3589429062 wait complete. 2024-04-26T20:26:31.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.278+0000 7f96f6a00640 1 --2- 172.21.15.43:0/2719079721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0038270 0x7f96f00eba00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009720 tx=0x7f96ec02a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2556de73bef2117c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.279+0000 7f96f6000640 1 -- 172.21.15.43:0/2719079721 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec037070 con 0x7f96f0038270 2024-04-26T20:26:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.279+0000 7f96f6000640 1 -- 172.21.15.43:0/2719079721 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec02adc0 con 0x7f96f0038270 2024-04-26T20:26:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.279+0000 7f96f6000640 1 -- 172.21.15.43:0/2719079721 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec032660 con 0x7f96f0038270 2024-04-26T20:26:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.279+0000 7f6b8f200640 1 Processor -- start 2024-04-26T20:26:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.279+0000 7f6b8f200640 1 -- start start 2024-04-26T20:26:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0038270 msgr2=0x7f96f00eba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f96fe000640 1 --2- 172.21.15.43:0/2719079721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0038270 0x7f96f00eba00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009720 tx=0x7f96ec02a0b0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 0x7f6b901574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90141540 0x7f6b901e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b900b31d0 con 0x7f6b90073300 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9006f590 con 0x7f6b900b43d0 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.280+0000 7f6b8f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6b900b3350 con 0x7f6b90141540 2024-04-26T20:26:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 shutdown_connections 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f6b8de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f6b8de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54816/0 (socket says 172.21.15.43:54816) 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f6b8de00640 1 -- 172.21.15.43:0/3338577005 learned_addr learned my addr 172.21.15.43:0/3338577005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f6b8e800640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90141540 0x7f6b901e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f6b8d400640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 0x7f6b901574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 -- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90141540 msgr2=0x7f6b901e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90141540 0x7f6b901e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 -- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 msgr2=0x7f6b901574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 0x7f6b901574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 -- 172.21.15.43:0/3338577005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b90140b80 con 0x7f6b90073300 2024-04-26T20:26:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.281+0000 7f96fe000640 1 --2- 172.21.15.43:0/2719079721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0038270 0x7f96f00eba00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8d400640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 0x7f6b901574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 >> 172.21.15.43:0/2719079721 conn(0x7f96f00559d0 msgr2=0x7f96f0055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.282+0000 7f6b8de00640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c460 tx=0x7f6b7c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.283+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c017070 con 0x7f6b90073300 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.283+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b901405c0 con 0x7f6b90073300 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.283+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c0035f0 con 0x7f6b90073300 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.283+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c003a20 con 0x7f6b90073300 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.284+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9013ec40 con 0x7f6b90073300 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.285+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 shutdown_connections 2024-04-26T20:26:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.285+0000 7f96fe000640 1 -- 172.21.15.43:0/2719079721 wait complete. 2024-04-26T20:26:31.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.286+0000 7fc1d9600640 1 Processor -- start 2024-04-26T20:26:31.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.286+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b90069350 con 0x7f6b90073300 2024-04-26T20:26:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.287+0000 7fc1d9600640 1 -- start start 2024-04-26T20:26:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.288+0000 7fdcf7200640 1 Processor -- start 2024-04-26T20:26:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.288+0000 7fdcf7200640 1 -- start start 2024-04-26T20:26:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.288+0000 7f96fe000640 1 Processor -- start 2024-04-26T20:26:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.287+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b7c01a070 con 0x7f6b90073300 2024-04-26T20:26:31.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.288+0000 7f96fe000640 1 -- start start 2024-04-26T20:26:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.289+0000 7f6b86000640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 0x7f6b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.289+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6b7c08b160 con 0x7f6b90073300 2024-04-26T20:26:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.289+0000 7fc1d9600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1d40d3480 0x7fc1d40c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fc1d9600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d40cf860 con 0x7fc1d40d1d90 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fc1d9600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d40cfb00 con 0x7fc1d4162880 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fc1d9600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d40d3090 con 0x7fc1d40d3480 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7f6b8d400640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 0x7f6b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fdcf7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fdcf7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdcf00b7580 0x7fdcf015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7f96fe000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 0x7f96f00dea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fdcf7200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00d3a10 con 0x7fdcf015ef00 2024-04-26T20:26:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fdcf7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00c8830 con 0x7fdcf00b7580 2024-04-26T20:26:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.290+0000 7fdcf7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00d3cb0 con 0x7fdcf00b71a0 2024-04-26T20:26:31.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7f96fe000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7f6b8d400640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 0x7f6b7006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b9015ad80 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7fdcf5200640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdcf015ef00 0x7fdcf0162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:44290/0 (socket says 172.21.15.43:44290) 2024-04-26T20:26:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7f96fe000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 0x7f96f00e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7fdcf5200640 1 -- 172.21.15.43:0/549837446 learned_addr learned my addr 172.21.15.43:0/549837446 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7fdcef400640 1 --2- 172.21.15.43:0/549837446 >> v2:172.21.15.164:3300/0 conn(0x7fdcf00b7580 0x7fdcf015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7fdcefe00640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7fc1d3400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc1d40d1d90 0x7fc1d4162170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41544/0 (socket says 172.21.15.43:41544) 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.291+0000 7fc1d3400640 1 -- 172.21.15.43:0/2876331902 learned_addr learned my addr 172.21.15.43:0/2876331902 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7f96f7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 0x7f96f00dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7f96f6a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7f96f6a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:55542/0 (socket says 172.21.15.43:55542) 2024-04-26T20:26:31.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7f96f7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 0x7f96f00dea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46714/0 (socket says 172.21.15.43:46714) 2024-04-26T20:26:31.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.292+0000 7fc1d3e00640 1 --2- 172.21.15.43:0/2876331902 >> v2:172.21.15.43:3300/0 conn(0x7fc1d40d3480 0x7fc1d40c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fdcef400640 1 -- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 msgr2=0x7fdcf00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fc1d3e00640 1 -- 172.21.15.43:0/2876331902 >> v1:172.21.15.164:6789/0 conn(0x7fc1d4162880 legacy=0x7fc1d40cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fc1d3e00640 1 -- 172.21.15.43:0/2876331902 >> v1:172.21.15.43:6789/0 conn(0x7fc1d40d1d90 legacy=0x7fc1d4162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fc1d3e00640 1 -- 172.21.15.43:0/2876331902 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d40d2ad0 con 0x7fc1d40d3480 2024-04-26T20:26:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fdcef400640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fdcef400640 1 -- 172.21.15.43:0/549837446 >> v1:172.21.15.164:6789/0 conn(0x7fdcf015ef00 legacy=0x7fdcf0162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7f96f6a00640 1 -- 172.21.15.43:0/3468923717 learned_addr learned my addr 172.21.15.43:0/3468923717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00ec190 con 0x7f96f00e7810 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00587b0 con 0x7f96f0006190 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f96f0058cd0 con 0x7f96f0038270 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fdceea00640 1 -- 172.21.15.43:0/549837446 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 25024486 0 0) 0x7fdcf00d3a10 con 0x7fdcf015ef00 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.293+0000 7fdcef400640 1 -- 172.21.15.43:0/549837446 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00c8af0 con 0x7fdcf00b7580 2024-04-26T20:26:31.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f7e00640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 0x7f96f00e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7fdade000640 1 Processor -- start 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f6a00640 1 -- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 msgr2=0x7f96f00dea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f6a00640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 0x7f96f00dea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f6a00640 1 -- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 msgr2=0x7f96f00e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f6a00640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 0x7f96f00e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f6a00640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f0185c00 con 0x7f96f0038270 2024-04-26T20:26:31.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7fc1d3e00640 1 --2- 172.21.15.43:0/2876331902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d40c87a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00097a0 tx=0x7fc1c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b52b397b7e49a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7fdade000640 1 -- start start 2024-04-26T20:26:31.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.294+0000 7f96f7e00640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 0x7f96f00e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:31.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.295+0000 7f96f6a00640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f96ec02a5d0 tx=0x7f96ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.295+0000 7fdcef400640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf00b7580 0x7fdcf015e9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0097a0 tx=0x7fdcdc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4575ae214fd3d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.296+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec042020 con 0x7f96f0038270 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.296+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec03d670 con 0x7f96f0038270 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.296+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec037040 con 0x7f96f0038270 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.296+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f00e8180 con 0x7f96f0038270 2024-04-26T20:26:31.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.296+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f00e8740 con 0x7f96f0038270 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdad814ceb0 0x7fdad80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad80d1570 0x7fdad80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 0x7fdad806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdad814d290 con 0x7fdad814ceb0 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdad80b5140 con 0x7fdad80cfc90 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdade000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdad80b5420 con 0x7fdad80d1570 2024-04-26T20:26:31.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdad7400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdad814ceb0 0x7fdad80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdceea00640 1 -- 172.21.15.43:0/549837446 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc037070 con 0x7fdcf00b7580 2024-04-26T20:26:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdceea00640 1 -- 172.21.15.43:0/549837446 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc02ae50 con 0x7fdcf00b7580 2024-04-26T20:26:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdceea00640 1 -- 172.21.15.43:0/549837446 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc032660 con 0x7fdcf00b7580 2024-04-26T20:26:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdad7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 0x7fdad806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdad7e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 0x7fdad806dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46718/0 (socket says 172.21.15.43:46718) 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.297+0000 7fdad7e00640 1 -- 172.21.15.43:0/1956602827 learned_addr learned my addr 172.21.15.43:0/1956602827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fdad6a00640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad80d1570 0x7fdad80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fc1d2a00640 1 -- 172.21.15.43:0/2876331902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0037070 con 0x7fc1d40d3480 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fc1d2a00640 1 -- 172.21.15.43:0/2876331902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c0032440 con 0x7fc1d40d3480 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fc1d2a00640 1 -- 172.21.15.43:0/2876331902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0037070 con 0x7fc1d40d3480 2024-04-26T20:26:31.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f00057d0 con 0x7f96f0038270 2024-04-26T20:26:31.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 -- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf00b7580 msgr2=0x7fdcf015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf00b7580 0x7fdcf015e9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0097a0 tx=0x7fdcdc02a100 comp rx=0 tx=0).stop 2024-04-26T20:26:31.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 -- 172.21.15.43:0/549837446 shutdown_connections 2024-04-26T20:26:31.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf00b7580 0x7fdcf015e9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 --2- 172.21.15.43:0/549837446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdcf7200640 1 -- 172.21.15.43:0/549837446 >> 172.21.15.43:0/549837446 conn(0x7fdcf00d4d70 msgr2=0x7fdcf00d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.300+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b7c058bf0 con 0x7f6b90073300 2024-04-26T20:26:31.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fdad7400640 1 -- 172.21.15.43:0/1956602827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad80d1570 msgr2=0x7fdad80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fdad7400640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad80d1570 0x7fdad80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ec045020 con 0x7f96f0038270 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 -- 172.21.15.43:0/2876331902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 msgr2=0x7fc1d40c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 --2- 172.21.15.43:0/2876331902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d40c87a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00097a0 tx=0x7fc1c002a230 comp rx=0 tx=0).stop 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fdad7400640 1 -- 172.21.15.43:0/1956602827 >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 msgr2=0x7fdad806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.298+0000 7fdad7400640 1 --2- 172.21.15.43:0/1956602827 >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 0x7fdad806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.299+0000 7fdad7400640 1 -- 172.21.15.43:0/1956602827 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad80b5700 con 0x7fdad814ceb0 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.300+0000 7fdcf7200640 1 -- 172.21.15.43:0/549837446 shutdown_connections 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.300+0000 7fdcf7200640 1 -- 172.21.15.43:0/549837446 wait complete. 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 -- 172.21.15.43:0/2876331902 shutdown_connections 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 --2- 172.21.15.43:0/2876331902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d40c87a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 -- 172.21.15.43:0/2876331902 >> 172.21.15.43:0/2876331902 conn(0x7fc1d40d4c50 msgr2=0x7fc1d40d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 -- 172.21.15.43:0/2876331902 shutdown_connections 2024-04-26T20:26:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fc1d9600640 1 -- 172.21.15.43:0/2876331902 wait complete. 2024-04-26T20:26:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.301+0000 7fdad7400640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad814ceb0 0x7fdad80b4a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdad8166f50 tx=0x7fdac802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85badca9c6f0da36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.302+0000 7fdad6000640 1 -- 172.21.15.43:0/1956602827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8037070 con 0x7fdad814ceb0 2024-04-26T20:26:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.302+0000 7f96eb400640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.302+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f96ec0ac690 con 0x7f96f0038270 2024-04-26T20:26:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.302+0000 7fdad6000640 1 -- 172.21.15.43:0/1956602827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8032440 con 0x7fdad814ceb0 2024-04-26T20:26:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.302+0000 7fdad6000640 1 -- 172.21.15.43:0/1956602827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac802ae60 con 0x7fdad814ceb0 2024-04-26T20:26:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.303+0000 7f96f7400640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.303+0000 7fdcf7200640 1 Processor -- start 2024-04-26T20:26:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdcf7200640 1 -- start start 2024-04-26T20:26:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 -- 172.21.15.43:0/1956602827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad814ceb0 msgr2=0x7fdad80b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad814ceb0 0x7fdad80b4a60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdad8166f50 tx=0x7fdac802a110 comp rx=0 tx=0).stop 2024-04-26T20:26:31.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fc1d9600640 1 Processor -- start 2024-04-26T20:26:31.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7f96f7400640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 0x7f96d806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96e4001030 tx=0x7f96e4007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 -- 172.21.15.43:0/1956602827 shutdown_connections 2024-04-26T20:26:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fc1d9600640 1 -- start start 2024-04-26T20:26:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 --2- 172.21.15.43:0/1956602827 >> v2:172.21.15.164:3300/0 conn(0x7fdad80cfc90 0x7fdad806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad80d1570 0x7fdad80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 --2- 172.21.15.43:0/1956602827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad814ceb0 0x7fdad80b4a60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.304+0000 7fdade000640 1 -- 172.21.15.43:0/1956602827 >> 172.21.15.43:0/1956602827 conn(0x7fdad80b2900 msgr2=0x7fdad80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.305+0000 7fdade000640 1 -- 172.21.15.43:0/1956602827 shutdown_connections 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.305+0000 7fdade000640 1 -- 172.21.15.43:0/1956602827 wait complete. 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.305+0000 7fc1d9600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 0x7fc1d41f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.305+0000 7fc1d9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fc1d9600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1d4162880 0x7fc1d40cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fc1d9600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d40d5770 con 0x7fc1d40d3480 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fc1d9600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d40d5b50 con 0x7fc1d40d1d90 2024-04-26T20:26:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fc1d9600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d40d58f0 con 0x7fc1d4162880 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf01fd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 0x7fdcf006d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf015f390 con 0x7fdcf00b7580 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00c90b0 con 0x7fdcf015ef00 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf015f510 con 0x7fdcf00b71a0 2024-04-26T20:26:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.306+0000 7fdcf5200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 0x7fdcf006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 0x7fc1d41f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fdcef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fdcef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54870/0 (socket says 172.21.15.43:54870) 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fdcef400640 1 -- 172.21.15.43:0/3735067380 learned_addr learned my addr 172.21.15.43:0/3735067380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fdcefe00640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf01fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcef400640 1 -- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 msgr2=0x7fdcf01fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcef400640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf01fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcef400640 1 -- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 msgr2=0x7fdcf006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcef400640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 0x7fdcf006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcef400640 1 -- 172.21.15.43:0/3735067380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00ba300 con 0x7fdcf00b7580 2024-04-26T20:26:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fdcf5200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 0x7fdcf006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fc1d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fc1d3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54856/0 (socket says 172.21.15.43:54856) 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fc1d3400640 1 -- 172.21.15.43:0/3110824718 learned_addr learned my addr 172.21.15.43:0/3110824718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.307+0000 7fc1d8c00640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1d4162880 0x7fc1d40cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fc1d3400640 1 -- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1d4162880 msgr2=0x7fc1d40cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fc1d3400640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1d4162880 0x7fc1d40cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.308+0000 7fc1d3400640 1 -- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 msgr2=0x7fc1d41f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fdade000640 1 Processor -- start 2024-04-26T20:26:31.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fc1d3400640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 0x7fc1d41f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fc1d3400640 1 -- 172.21.15.43:0/3110824718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d41f94d0 con 0x7fc1d40d3480 2024-04-26T20:26:31.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fdade000640 1 -- start start 2024-04-26T20:26:31.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fc1d3e00640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 0x7fc1d41f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fdcef400640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02a6f0 tx=0x7fdcdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fc1d3400640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c500 tx=0x7fc1c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.309+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc042020 con 0x7fdcf00b7580 2024-04-26T20:26:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.310+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc03d690 con 0x7fdcf00b7580 2024-04-26T20:26:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.310+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf01fd590 con 0x7fdcf00b7580 2024-04-26T20:26:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.310+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4005ce0 con 0x7fc1d40d3480 2024-04-26T20:26:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.310+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4007070 con 0x7fc1d40d3480 2024-04-26T20:26:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.310+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4007420 con 0x7fc1d40d3480 2024-04-26T20:26:31.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 0x7fdad80af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 0x7fdad80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad814ceb0 0x7fdad80c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d41f97b0 con 0x7fc1d40d3480 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdad80d19d0 con 0x7fdad80d1570 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdad80b5cc0 con 0x7fdad80cfc90 2024-04-26T20:26:31.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdade000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdad806e530 con 0x7fdad814ceb0 2024-04-26T20:26:31.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.311+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc037040 con 0x7fdcf00b7580 2024-04-26T20:26:31.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fdad6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 0x7fdad80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01585f0 con 0x7fdcf00b7580 2024-04-26T20:26:31.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fdad7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 0x7fdad80af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fdad7400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 0x7fdad80af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46750/0 (socket says 172.21.15.43:46750) 2024-04-26T20:26:31.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fdad7400640 1 -- 172.21.15.43:0/2414329537 learned_addr learned my addr 172.21.15.43:0/2414329537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:31.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.312+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d40cd040 con 0x7fc1d40d3480 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.313+0000 7fdad7e00640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad814ceb0 0x7fdad80c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fdad6a00640 1 -- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad814ceb0 msgr2=0x7fdad80c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fdad6a00640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad814ceb0 0x7fdad80c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fdad6a00640 1 -- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 msgr2=0x7fdad80af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fdad6a00640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 0x7fdad80af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d41f9cd0 con 0x7fc1d40d3480 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.314+0000 7fdad6a00640 1 -- 172.21.15.43:0/2414329537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad80ac990 con 0x7fdad80d1570 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.315+0000 7fdad6a00640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 0x7fdad80cc770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e850 tx=0x7fdac400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.316+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4017070 con 0x7fdad80d1570 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.316+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac400c940 con 0x7fdad80d1570 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.316+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400cd10 con 0x7fdad80d1570 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.316+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad80acc70 con 0x7fdad80d1570 2024-04-26T20:26:31.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.316+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcdc045020 con 0x7fdcf00b7580 2024-04-26T20:26:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.317+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad80ad230 con 0x7fdad80d1570 2024-04-26T20:26:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.318+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad8069350 con 0x7fdad80d1570 2024-04-26T20:26:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.318+0000 7fdce7e00640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.318+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1c40075c0 con 0x7fc1d40d3480 2024-04-26T20:26:31.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.318+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdcdc0abea0 con 0x7fdcf00b7580 2024-04-26T20:26:31.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.319+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0069350 con 0x7fdcf00b7580 2024-04-26T20:26:31.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.319+0000 7fdcefe00640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.320+0000 7fc1cbe00640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 0x7fc1b406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.320+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc1c408b940 con 0x7fc1d40d3480 2024-04-26T20:26:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.321+0000 7fdcefe00640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 0x7fdcd006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009ad0 tx=0x7fdce0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.321+0000 7fc1d3e00640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 0x7fc1b406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.321+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96ec04c030 con 0x7f96f0038270 2024-04-26T20:26:31.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.322+0000 7fc1d3e00640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 0x7fc1b406ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0034000 tx=0x7fc1c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.321+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdac401a070 con 0x7fdad80d1570 2024-04-26T20:26:31.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.322+0000 7fdacf400640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 0x7fdab806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.323+0000 7fdad7400640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 0x7fdab806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.324+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdac408bfb0 con 0x7fdad80d1570 2024-04-26T20:26:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.324+0000 7fdad7400640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 0x7fdab806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009560 tx=0x7fdac802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.330+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcdc07a380 con 0x7fdcf00b7580 2024-04-26T20:26:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.339+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1d41f9cd0 con 0x7fc1d40d3480 2024-04-26T20:26:31.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.348+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdac4059b30 con 0x7fdad80d1570 2024-04-26T20:26:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.621+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f01b00b1e30 con 0x7f01b00d1cf0 2024-04-26T20:26:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.622+0000 7f01acc00640 1 -- 172.21.15.43:0/3807447094 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01a8059fd0 con 0x7f01b00d1cf0 2024-04-26T20:26:31.625 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.626+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 msgr2=0x7f019006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.627+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 0x7f019006ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f019c0011e0 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.627+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 msgr2=0x7f01b0163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.627+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01a800efa0 tx=0x7f01a800c550 comp rx=0 tx=0).stop 2024-04-26T20:26:31.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.628+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 shutdown_connections 2024-04-26T20:26:31.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.628+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f019006c940 0x7f019006ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.628+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01b00d1cf0 0x7f01b0163170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.628+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01b00b4020 0x7f01b01653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.629+0000 7f01b6200640 1 --2- 172.21.15.43:0/3807447094 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01b00b3c40 0x7f01b00b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.629+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 >> 172.21.15.43:0/3807447094 conn(0x7f01b00d4c80 msgr2=0x7f01b00ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.629+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 shutdown_connections 2024-04-26T20:26:31.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.629+0000 7f01b6200640 1 -- 172.21.15.43:0/3807447094 wait complete. 2024-04-26T20:26:31.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2024-04-26T20:26:31.657 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.662+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f55280e0770 con 0x7f552804a0a0 2024-04-26T20:26:31.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.663+0000 7f5523e00640 1 -- 172.21.15.43:0/1569332977 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f551c059b40 con 0x7f552804a0a0 2024-04-26T20:26:31.665 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.667+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 msgr2=0x7f551006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.667+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 0x7f551006ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f552804a870 tx=0x7f5524009240 comp rx=0 tx=0).stop 2024-04-26T20:26:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.667+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 msgr2=0x7f5528009e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.667+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f551c00e9b0 tx=0x7f551c00ee70 comp rx=0 tx=0).stop 2024-04-26T20:26:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 shutdown_connections 2024-04-26T20:26:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f551006c9c0 0x7f551006ee80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5528054de0 0x7f552804afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55280e7a50 0x7f552804b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 --2- 172.21.15.43:0/1569332977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552804a0a0 0x7f5528009e40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 >> 172.21.15.43:0/1569332977 conn(0x7f5528056cd0 msgr2=0x7f55280e5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 shutdown_connections 2024-04-26T20:26:31.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.668+0000 7f5535a00640 1 -- 172.21.15.43:0/1569332977 wait complete. 2024-04-26T20:26:31.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.673+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdf50056bf0 con 0x7fdf500e3160 2024-04-26T20:26:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.674+0000 7fdf55600640 1 -- 172.21.15.43:0/3409775458 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf4405d990 con 0x7fdf500e3160 2024-04-26T20:26:31.678 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.677+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 msgr2=0x7fdf3806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.677+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 0x7fdf3806ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007fd0 tx=0x7fdf4c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.677+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 msgr2=0x7fdf500390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.677+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 0x7fdf500390f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e880 tx=0x7fdf4400ed40 comp rx=0 tx=0).stop 2024-04-26T20:26:31.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 shutdown_connections 2024-04-26T20:26:31.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf3806c9c0 0x7fdf3806ee80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf50053cb0 0x7fdf500e4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf500e3160 0x7fdf500390f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 --2- 172.21.15.43:0/3409775458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf5003c0a0 0x7fdf5004f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 >> 172.21.15.43:0/3409775458 conn(0x7fdf50058ee0 msgr2=0x7fdf500e74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 shutdown_connections 2024-04-26T20:26:31.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.678+0000 7fdf5f000640 1 -- 172.21.15.43:0/3409775458 wait complete. 2024-04-26T20:26:31.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2024-04-26T20:26:31.697 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2024-04-26T20:26:31.705 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.717+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6b900b3f80 con 0x7f6b90073300 2024-04-26T20:26:31.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.718+0000 7f6b86000640 1 -- 172.21.15.43:0/3338577005 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b7c05cef0 con 0x7f6b90073300 2024-04-26T20:26:31.722 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 msgr2=0x7f6b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 0x7f6b7006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b9015ad80 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 msgr2=0x7f6b90159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c460 tx=0x7f6b7c00c920 comp rx=0 tx=0).stop 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 shutdown_connections 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b7006c8f0 0x7f6b7006edb0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b9015ad80 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b90141540 0x7f6b901e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b900b43d0 0x7f6b901574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 --2- 172.21.15.43:0/3338577005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b90073300 0x7f6b90159090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.720+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 >> 172.21.15.43:0/3338577005 conn(0x7f6b900b26b0 msgr2=0x7f6b9015bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.721+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 shutdown_connections 2024-04-26T20:26:31.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.721+0000 7f6b8f200640 1 -- 172.21.15.43:0/3338577005 wait complete. 2024-04-26T20:26:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.724+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdad80b4d00 con 0x7fdad80d1570 2024-04-26T20:26:31.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.725+0000 7fdacf400640 1 -- 172.21.15.43:0/2414329537 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdac405de30 con 0x7fdad80d1570 2024-04-26T20:26:31.726 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 msgr2=0x7fdab806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 0x7fdab806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009560 tx=0x7fdac802e000 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 msgr2=0x7fdad80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 0x7fdad80cc770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdac400e850 tx=0x7fdac400ed10 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 shutdown_connections 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdab806c940 0x7fdab806ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad814ceb0 0x7fdad80c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdad80d1570 0x7fdad80cc770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 --2- 172.21.15.43:0/2414329537 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdad80cfc90 0x7fdad80af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 >> 172.21.15.43:0/2414329537 conn(0x7fdad80b2900 msgr2=0x7fdad8074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.727+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 shutdown_connections 2024-04-26T20:26:31.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.728+0000 7fdade000640 1 -- 172.21.15.43:0/2414329537 wait complete. 2024-04-26T20:26:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.733+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdcf008ee00 con 0x7fdcf00b7580 2024-04-26T20:26:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.734+0000 7fdce7e00640 1 -- 172.21.15.43:0/3735067380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdcdc079c80 con 0x7fdcf00b7580 2024-04-26T20:26:31.735 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 msgr2=0x7fdcd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 0x7fdcd006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009ad0 tx=0x7fdce0008040 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 msgr2=0x7fdcf0152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02a6f0 tx=0x7fdcdc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 shutdown_connections 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdcd006c8f0 0x7fdcd006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdcf015ef00 0x7fdcf006d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcf00b7580 0x7fdcf0152170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 --2- 172.21.15.43:0/3735067380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdcf00b71a0 0x7fdcf01fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.736+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 >> 172.21.15.43:0/3735067380 conn(0x7fdcf00d4d70 msgr2=0x7fdcf00ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.737+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 shutdown_connections 2024-04-26T20:26:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.737+0000 7fdcf7200640 1 -- 172.21.15.43:0/3735067380 wait complete. 2024-04-26T20:26:31.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.5 2024-04-26T20:26:31.744 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2024-04-26T20:26:31.751 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.1 2024-04-26T20:26:31.754 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.762+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f96f00e5d60 con 0x7f96f0038270 2024-04-26T20:26:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.763+0000 7f96eb400640 1 -- 172.21.15.43:0/3468923717 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96ec07a6e0 con 0x7f96f0038270 2024-04-26T20:26:31.764 INFO:teuthology.orchestra.run.smithi043.stdout:55834574860 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 msgr2=0x7f96d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 0x7f96d806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96e4001030 tx=0x7f96e4007d50 comp rx=0 tx=0).stop 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 msgr2=0x7f96f00501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f96ec02a5d0 tx=0x7f96ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 shutdown_connections 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96d806c8f0 0x7f96d806edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96f0038270 0x7f96f00501f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f00e7810 0x7f96f00e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 --2- 172.21.15.43:0/3468923717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96f0006190 0x7f96f00dea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 >> 172.21.15.43:0/3468923717 conn(0x7f96f00559d0 msgr2=0x7f96f003b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 shutdown_connections 2024-04-26T20:26:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.766+0000 7f96fe000640 1 -- 172.21.15.43:0/3468923717 wait complete. 2024-04-26T20:26:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.787+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc1d40cd240 con 0x7fc1d40d3480 2024-04-26T20:26:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.788+0000 7fc1cbe00640 1 -- 172.21.15.43:0/3110824718 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1d40cd240 con 0x7fc1d40d3480 2024-04-26T20:26:31.789 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-04-26T20:26:31.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.2 2024-04-26T20:26:31.793 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 msgr2=0x7fc1b406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 0x7fc1b406ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0034000 tx=0x7fc1c002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 msgr2=0x7fc1d41603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c500 tx=0x7fc1c400c9c0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 shutdown_connections 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1b406c940 0x7fc1b406ee00 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0034000 tx=0x7fc1c002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1d4162880 0x7fc1d40cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40d3480 0x7fc1d41603c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 --2- 172.21.15.43:0/3110824718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1d40d1d90 0x7fc1d41f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 >> 172.21.15.43:0/3110824718 conn(0x7fc1d40d4c50 msgr2=0x7fc1d40c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 shutdown_connections 2024-04-26T20:26:31.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:31.791+0000 7fc1d9600640 1 -- 172.21.15.43:0/3110824718 wait complete. 2024-04-26T20:26:31.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.6 2024-04-26T20:26:31.810 DEBUG:teuthology.parallel:result is None 2024-04-26T20:26:31.810 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:26:31.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.054+0000 7f0b27e00640 1 Processor -- start 2024-04-26T20:26:32.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.054+0000 7f0b27e00640 1 -- start start 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.055+0000 7f0b27e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b2815f170 0x7f0b280d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.055+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 0x7f0b280b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.055+0000 7f0b27e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b280d0f10 con 0x7f0b280d5850 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.055+0000 7f0b27e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0b280d0c30 con 0x7f0b2815f170 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.055+0000 7f0b27e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0b280b42a0 con 0x7f0b280d3870 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0b280d5850 0x7f0b280d3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41550/0 (socket says 172.21.15.43:41550) 2024-04-26T20:26:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26000640 1 -- 172.21.15.43:0/3526481709 learned_addr learned my addr 172.21.15.43:0/3526481709 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26a00640 1 --2- 172.21.15.43:0/3526481709 >> v2:172.21.15.43:3300/0 conn(0x7f0b2815f170 0x7f0b280d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b27400640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 0x7f0b280b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b25600640 1 -- 172.21.15.43:0/3526481709 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001637174 0 0) 0x7f0b280d0f10 con 0x7f0b280d5850 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b25600640 1 -- 172.21.15.43:0/3526481709 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b0c003520 con 0x7f0b280d5850 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26a00640 1 -- 172.21.15.43:0/3526481709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 msgr2=0x7f0b280b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26a00640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 0x7f0b280b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26a00640 1 -- 172.21.15.43:0/3526481709 >> v1:172.21.15.43:6789/0 conn(0x7f0b280d5850 legacy=0x7f0b280d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b26a00640 1 -- 172.21.15.43:0/3526481709 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b280b4580 con 0x7f0b2815f170 2024-04-26T20:26:32.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.056+0000 7f0b27400640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 0x7f0b280b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b26a00640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2815f170 0x7f0b280d2a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b180097a0 tx=0x7f0b1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb9968fdbb289e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b25600640 1 -- 172.21.15.43:0/3526481709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b2815f170 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b25600640 1 -- 172.21.15.43:0/3526481709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b2815f170 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b25600640 1 -- 172.21.15.43:0/3526481709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b2815f170 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b27e00640 1 -- 172.21.15.43:0/3526481709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2815f170 msgr2=0x7f0b280d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.058+0000 7f0b27e00640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2815f170 0x7f0b280d2a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b180097a0 tx=0x7f0b1802a180 comp rx=0 tx=0).stop 2024-04-26T20:26:32.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 -- 172.21.15.43:0/3526481709 shutdown_connections 2024-04-26T20:26:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d3870 0x7f0b280b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 --2- 172.21.15.43:0/3526481709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2815f170 0x7f0b280d2a80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 -- 172.21.15.43:0/3526481709 >> 172.21.15.43:0/3526481709 conn(0x7f0b280d4ca0 msgr2=0x7f0b280d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 -- 172.21.15.43:0/3526481709 shutdown_connections 2024-04-26T20:26:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.059+0000 7f0b27e00640 1 -- 172.21.15.43:0/3526481709 wait complete. 2024-04-26T20:26:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.061+0000 7f0b27e00640 1 Processor -- start 2024-04-26T20:26:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.061+0000 7f0b27e00640 1 -- start start 2024-04-26T20:26:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d5850 0x7f0b280ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b2815f170 0x7f0b280a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280d0850 con 0x7f0b280d3870 2024-04-26T20:26:32.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2815f550 con 0x7f0b2815f170 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.062+0000 7f0b27e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0b280d09d0 con 0x7f0b280d5850 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54900/0 (socket says 172.21.15.43:54900) 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 -- 172.21.15.43:0/929056750 learned_addr learned my addr 172.21.15.43:0/929056750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 -- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d5850 msgr2=0x7f0b280ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d5850 0x7f0b280ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 -- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b2815f170 msgr2=0x7f0b280a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b2815f170 0x7f0b280a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 -- 172.21.15.43:0/929056750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2815cb10 con 0x7f0b280d3870 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.063+0000 7f0b26a00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b18035040 tx=0x7f0b180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.064+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037040 con 0x7f0b280d3870 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.064+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18030500 con 0x7f0b280d3870 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.064+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030910 con 0x7f0b280d3870 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.064+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2815f800 con 0x7f0b280d3870 2024-04-26T20:26:32.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.064+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2815fdc0 con 0x7f0b280d3870 2024-04-26T20:26:32.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.065+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b28069a60 con 0x7f0b280d3870 2024-04-26T20:26:32.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.066+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b18044020 con 0x7f0b280d3870 2024-04-26T20:26:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.067+0000 7f0b1ea00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 0x7f0b0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.067+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0b180ac1c0 con 0x7f0b280d3870 2024-04-26T20:26:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.067+0000 7f0b26000640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 0x7f0b0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.068+0000 7f0b26000640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 0x7f0b0406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000f60 tx=0x7f0b100078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.071+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b1807a210 con 0x7f0b280d3870 2024-04-26T20:26:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.262+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b2815cdf0 con 0x7f0b0406c920 2024-04-26T20:26:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.287+0000 7f0b1ea00640 1 -- 172.21.15.43:0/929056750 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f0b10067310 con 0x7f0b0406c920 2024-04-26T20:26:32.293 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:26:32.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 msgr2=0x7f0b0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 0x7f0b0406ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000f60 tx=0x7f0b100078f0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 msgr2=0x7f0b2815c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b18035040 tx=0x7f0b180046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 shutdown_connections 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0b0406c920 0x7f0b0406ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0b280d5850 0x7f0b280ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b2815f170 0x7f0b280a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 --2- 172.21.15.43:0/929056750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b280d3870 0x7f0b2815c480 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.293+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 >> 172.21.15.43:0/929056750 conn(0x7f0b280d4ca0 msgr2=0x7f0b280d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.294+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 shutdown_connections 2024-04-26T20:26:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.294+0000 7f0b27e00640 1 -- 172.21.15.43:0/929056750 wait complete. 2024-04-26T20:26:32.313 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-26T20:26:30.869891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227728,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746992,"statfs":{"total":773094113280,"available":772860919808,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005603"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347756+0000","last_change":"2024-04-26T20:25:56.984031+0000","last_active":"2024-04-26T20:26:03.347756+0000","last_peered":"2024-04-26T20:26:03.347756+0000","last_clean":"2024-04-26T20:26:03.347756+0000","last_became_active":"2024-04-26T20:25:56.983640+0000","last_became_peered":"2024-04-26T20:25:56.983640+0000","last_unstale":"2024-04-26T20:26:03.347756+0000","last_undegraded":"2024-04-26T20:26:03.347756+0000","last_fullsized":"2024-04-26T20:26:03.347756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:04.709694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094688+0000","last_change":"2024-04-26T20:25:57.999359+0000","last_active":"2024-04-26T20:26:03.094688+0000","last_peered":"2024-04-26T20:26:03.094688+0000","last_clean":"2024-04-26T20:26:03.094688+0000","last_became_active":"2024-04-26T20:25:57.999124+0000","last_became_peered":"2024-04-26T20:25:57.999124+0000","last_unstale":"2024-04-26T20:26:03.094688+0000","last_undegraded":"2024-04-26T20:26:03.094688+0000","last_fullsized":"2024-04-26T20:26:03.094688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092925+0000","last_change":"2024-04-26T20:26:02.417831+0000","last_active":"2024-04-26T20:26:03.092925+0000","last_peered":"2024-04-26T20:26:03.092925+0000","last_clean":"2024-04-26T20:26:03.092925+0000","last_became_active":"2024-04-26T20:26:02.417256+0000","last_became_peered":"2024-04-26T20:26:02.417256+0000","last_unstale":"2024-04-26T20:26:03.092925+0000","last_undegraded":"2024-04-26T20:26:03.092925+0000","last_fullsized":"2024-04-26T20:26:03.092925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094948+0000","last_change":"2024-04-26T20:25:56.981647+0000","last_active":"2024-04-26T20:26:03.094948+0000","last_peered":"2024-04-26T20:26:03.094948+0000","last_clean":"2024-04-26T20:26:03.094948+0000","last_became_active":"2024-04-26T20:25:56.980997+0000","last_became_peered":"2024-04-26T20:25:56.980997+0000","last_unstale":"2024-04-26T20:26:03.094948+0000","last_undegraded":"2024-04-26T20:26:03.094948+0000","last_fullsized":"2024-04-26T20:26:03.094948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347273+0000","last_change":"2024-04-26T20:25:58.006366+0000","last_active":"2024-04-26T20:26:03.347273+0000","last_peered":"2024-04-26T20:26:03.347273+0000","last_clean":"2024-04-26T20:26:03.347273+0000","last_became_active":"2024-04-26T20:25:58.005106+0000","last_became_peered":"2024-04-26T20:25:58.005106+0000","last_unstale":"2024-04-26T20:26:03.347273+0000","last_undegraded":"2024-04-26T20:26:03.347273+0000","last_fullsized":"2024-04-26T20:26:03.347273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050650+0000","last_change":"2024-04-26T20:26:02.077388+0000","last_active":"2024-04-26T20:26:03.050650+0000","last_peered":"2024-04-26T20:26:03.050650+0000","last_clean":"2024-04-26T20:26:03.050650+0000","last_became_active":"2024-04-26T20:26:02.076618+0000","last_became_peered":"2024-04-26T20:26:02.076618+0000","last_unstale":"2024-04-26T20:26:03.050650+0000","last_undegraded":"2024-04-26T20:26:03.050650+0000","last_fullsized":"2024-04-26T20:26:03.050650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:59.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348208+0000","last_change":"2024-04-26T20:25:56.993261+0000","last_active":"2024-04-26T20:26:03.348208+0000","last_peered":"2024-04-26T20:26:03.348208+0000","last_clean":"2024-04-26T20:26:03.348208+0000","last_became_active":"2024-04-26T20:25:56.992657+0000","last_became_peered":"2024-04-26T20:25:56.992657+0000","last_unstale":"2024-04-26T20:26:03.348208+0000","last_undegraded":"2024-04-26T20:26:03.348208+0000","last_fullsized":"2024-04-26T20:26:03.348208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346205+0000","last_change":"2024-04-26T20:25:58.008848+0000","last_active":"2024-04-26T20:26:03.346205+0000","last_peered":"2024-04-26T20:26:03.346205+0000","last_clean":"2024-04-26T20:26:03.346205+0000","last_became_active":"2024-04-26T20:25:58.008316+0000","last_became_peered":"2024-04-26T20:25:58.008316+0000","last_unstale":"2024-04-26T20:26:03.346205+0000","last_undegraded":"2024-04-26T20:26:03.346205+0000","last_fullsized":"2024-04-26T20:26:03.346205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051568+0000","last_change":"2024-04-26T20:26:02.080666+0000","last_active":"2024-04-26T20:26:03.051568+0000","last_peered":"2024-04-26T20:26:03.051568+0000","last_clean":"2024-04-26T20:26:03.051568+0000","last_became_active":"2024-04-26T20:26:02.080286+0000","last_became_peered":"2024-04-26T20:26:02.080286+0000","last_unstale":"2024-04-26T20:26:03.051568+0000","last_undegraded":"2024-04-26T20:26:03.051568+0000","last_fullsized":"2024-04-26T20:26:03.051568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347297+0000","last_change":"2024-04-26T20:25:56.993036+0000","last_active":"2024-04-26T20:26:03.347297+0000","last_peered":"2024-04-26T20:26:03.347297+0000","last_clean":"2024-04-26T20:26:03.347297+0000","last_became_active":"2024-04-26T20:25:56.992712+0000","last_became_peered":"2024-04-26T20:25:56.992712+0000","last_unstale":"2024-04-26T20:26:03.347297+0000","last_undegraded":"2024-04-26T20:26:03.347297+0000","last_fullsized":"2024-04-26T20:26:03.347297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:11.781113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347210+0000","last_change":"2024-04-26T20:25:58.005509+0000","last_active":"2024-04-26T20:26:03.347210+0000","last_peered":"2024-04-26T20:26:03.347210+0000","last_clean":"2024-04-26T20:26:03.347210+0000","last_became_active":"2024-04-26T20:25:58.005266+0000","last_became_peered":"2024-04-26T20:25:58.005266+0000","last_unstale":"2024-04-26T20:26:03.347210+0000","last_undegraded":"2024-04-26T20:26:03.347210+0000","last_fullsized":"2024-04-26T20:26:03.347210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050541+0000","last_change":"2024-04-26T20:26:02.076146+0000","last_active":"2024-04-26T20:26:03.050541+0000","last_peered":"2024-04-26T20:26:03.050541+0000","last_clean":"2024-04-26T20:26:03.050541+0000","last_became_active":"2024-04-26T20:26:02.075597+0000","last_became_peered":"2024-04-26T20:26:02.075597+0000","last_unstale":"2024-04-26T20:26:03.050541+0000","last_undegraded":"2024-04-26T20:26:03.050541+0000","last_fullsized":"2024-04-26T20:26:03.050541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:49.620083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349354+0000","last_change":"2024-04-26T20:25:57.995411+0000","last_active":"2024-04-26T20:26:03.349354+0000","last_peered":"2024-04-26T20:26:03.349354+0000","last_clean":"2024-04-26T20:26:03.349354+0000","last_became_active":"2024-04-26T20:25:57.993865+0000","last_became_peered":"2024-04-26T20:25:57.993865+0000","last_unstale":"2024-04-26T20:26:03.349354+0000","last_undegraded":"2024-04-26T20:26:03.349354+0000","last_fullsized":"2024-04-26T20:26:03.349354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347802+0000","last_change":"2024-04-26T20:25:56.986121+0000","last_active":"2024-04-26T20:26:03.347802+0000","last_peered":"2024-04-26T20:26:03.347802+0000","last_clean":"2024-04-26T20:26:03.347802+0000","last_became_active":"2024-04-26T20:25:56.985836+0000","last_became_peered":"2024-04-26T20:25:56.985836+0000","last_unstale":"2024-04-26T20:26:03.347802+0000","last_undegraded":"2024-04-26T20:26:03.347802+0000","last_fullsized":"2024-04-26T20:26:03.347802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:01.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049987+0000","last_change":"2024-04-26T20:26:02.079776+0000","last_active":"2024-04-26T20:26:03.049987+0000","last_peered":"2024-04-26T20:26:03.049987+0000","last_clean":"2024-04-26T20:26:03.049987+0000","last_became_active":"2024-04-26T20:26:02.079355+0000","last_became_peered":"2024-04-26T20:26:02.079355+0000","last_unstale":"2024-04-26T20:26:03.049987+0000","last_undegraded":"2024-04-26T20:26:03.049987+0000","last_fullsized":"2024-04-26T20:26:03.049987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:32.933389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093856+0000","last_change":"2024-04-26T20:25:57.990924+0000","last_active":"2024-04-26T20:26:03.093856+0000","last_peered":"2024-04-26T20:26:03.093856+0000","last_clean":"2024-04-26T20:26:03.093856+0000","last_became_active":"2024-04-26T20:25:57.990615+0000","last_became_peered":"2024-04-26T20:25:57.990615+0000","last_unstale":"2024-04-26T20:26:03.093856+0000","last_undegraded":"2024-04-26T20:26:03.093856+0000","last_fullsized":"2024-04-26T20:26:03.093856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345060+0000","last_change":"2024-04-26T20:25:56.985786+0000","last_active":"2024-04-26T20:26:03.345060+0000","last_peered":"2024-04-26T20:26:03.345060+0000","last_clean":"2024-04-26T20:26:03.345060+0000","last_became_active":"2024-04-26T20:25:56.981498+0000","last_became_peered":"2024-04-26T20:25:56.981498+0000","last_unstale":"2024-04-26T20:26:03.345060+0000","last_undegraded":"2024-04-26T20:26:03.345060+0000","last_fullsized":"2024-04-26T20:26:03.345060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049606+0000","last_change":"2024-04-26T20:26:02.075573+0000","last_active":"2024-04-26T20:26:03.049606+0000","last_peered":"2024-04-26T20:26:03.049606+0000","last_clean":"2024-04-26T20:26:03.049606+0000","last_became_active":"2024-04-26T20:26:02.075012+0000","last_became_peered":"2024-04-26T20:26:02.075012+0000","last_unstale":"2024-04-26T20:26:03.049606+0000","last_undegraded":"2024-04-26T20:26:03.049606+0000","last_fullsized":"2024-04-26T20:26:03.049606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.401666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093255+0000","last_change":"2024-04-26T20:25:58.011840+0000","last_active":"2024-04-26T20:26:03.093255+0000","last_peered":"2024-04-26T20:26:03.093255+0000","last_clean":"2024-04-26T20:26:03.093255+0000","last_became_active":"2024-04-26T20:25:58.011346+0000","last_became_peered":"2024-04-26T20:25:58.011346+0000","last_unstale":"2024-04-26T20:26:03.093255+0000","last_undegraded":"2024-04-26T20:26:03.093255+0000","last_fullsized":"2024-04-26T20:26:03.093255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344179+0000","last_change":"2024-04-26T20:25:56.986011+0000","last_active":"2024-04-26T20:26:03.344179+0000","last_peered":"2024-04-26T20:26:03.344179+0000","last_clean":"2024-04-26T20:26:03.344179+0000","last_became_active":"2024-04-26T20:25:56.985728+0000","last_became_peered":"2024-04-26T20:25:56.985728+0000","last_unstale":"2024-04-26T20:26:03.344179+0000","last_undegraded":"2024-04-26T20:26:03.344179+0000","last_fullsized":"2024-04-26T20:26:03.344179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347438+0000","last_change":"2024-04-26T20:26:02.401041+0000","last_active":"2024-04-26T20:26:03.347438+0000","last_peered":"2024-04-26T20:26:03.347438+0000","last_clean":"2024-04-26T20:26:03.347438+0000","last_became_active":"2024-04-26T20:26:02.400108+0000","last_became_peered":"2024-04-26T20:26:02.400108+0000","last_unstale":"2024-04-26T20:26:03.347438+0000","last_undegraded":"2024-04-26T20:26:03.347438+0000","last_fullsized":"2024-04-26T20:26:03.347438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049644+0000","last_change":"2024-04-26T20:25:57.985787+0000","last_active":"2024-04-26T20:26:03.049644+0000","last_peered":"2024-04-26T20:26:03.049644+0000","last_clean":"2024-04-26T20:26:03.049644+0000","last_became_active":"2024-04-26T20:25:57.985432+0000","last_became_peered":"2024-04-26T20:25:57.985432+0000","last_unstale":"2024-04-26T20:26:03.049644+0000","last_undegraded":"2024-04-26T20:26:03.049644+0000","last_fullsized":"2024-04-26T20:26:03.049644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:37.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345394+0000","last_change":"2024-04-26T20:25:56.983944+0000","last_active":"2024-04-26T20:26:03.345394+0000","last_peered":"2024-04-26T20:26:03.345394+0000","last_clean":"2024-04-26T20:26:03.345394+0000","last_became_active":"2024-04-26T20:25:56.983642+0000","last_became_peered":"2024-04-26T20:25:56.983642+0000","last_unstale":"2024-04-26T20:26:03.345394+0000","last_undegraded":"2024-04-26T20:26:03.345394+0000","last_fullsized":"2024-04-26T20:26:03.345394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:12.976926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050080+0000","last_change":"2024-04-26T20:26:02.082133+0000","last_active":"2024-04-26T20:26:03.050080+0000","last_peered":"2024-04-26T20:26:03.050080+0000","last_clean":"2024-04-26T20:26:03.050080+0000","last_became_active":"2024-04-26T20:26:02.081547+0000","last_became_peered":"2024-04-26T20:26:02.081547+0000","last_unstale":"2024-04-26T20:26:03.050080+0000","last_undegraded":"2024-04-26T20:26:03.050080+0000","last_fullsized":"2024-04-26T20:26:03.050080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:39.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347003+0000","last_change":"2024-04-26T20:25:57.989909+0000","last_active":"2024-04-26T20:26:03.347003+0000","last_peered":"2024-04-26T20:26:03.347003+0000","last_clean":"2024-04-26T20:26:03.347003+0000","last_became_active":"2024-04-26T20:25:57.989626+0000","last_became_peered":"2024-04-26T20:25:57.989626+0000","last_unstale":"2024-04-26T20:26:03.347003+0000","last_undegraded":"2024-04-26T20:26:03.347003+0000","last_fullsized":"2024-04-26T20:26:03.347003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351956+0000","last_change":"2024-04-26T20:25:56.985557+0000","last_active":"2024-04-26T20:26:03.351956+0000","last_peered":"2024-04-26T20:26:03.351956+0000","last_clean":"2024-04-26T20:26:03.351956+0000","last_became_active":"2024-04-26T20:25:56.985217+0000","last_became_peered":"2024-04-26T20:25:56.985217+0000","last_unstale":"2024-04-26T20:26:03.351956+0000","last_undegraded":"2024-04-26T20:26:03.351956+0000","last_fullsized":"2024-04-26T20:26:03.351956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050981+0000","last_change":"2024-04-26T20:26:02.405217+0000","last_active":"2024-04-26T20:26:03.050981+0000","last_peered":"2024-04-26T20:26:03.050981+0000","last_clean":"2024-04-26T20:26:03.050981+0000","last_became_active":"2024-04-26T20:26:02.404124+0000","last_became_peered":"2024-04-26T20:26:02.404124+0000","last_unstale":"2024-04-26T20:26:03.050981+0000","last_undegraded":"2024-04-26T20:26:03.050981+0000","last_fullsized":"2024-04-26T20:26:03.050981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:29.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095109+0000","last_change":"2024-04-26T20:25:57.999075+0000","last_active":"2024-04-26T20:26:03.095109+0000","last_peered":"2024-04-26T20:26:03.095109+0000","last_clean":"2024-04-26T20:26:03.095109+0000","last_became_active":"2024-04-26T20:25:57.998666+0000","last_became_peered":"2024-04-26T20:25:57.998666+0000","last_unstale":"2024-04-26T20:26:03.095109+0000","last_undegraded":"2024-04-26T20:26:03.095109+0000","last_fullsized":"2024-04-26T20:26:03.095109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346736+0000","last_change":"2024-04-26T20:25:56.977150+0000","last_active":"2024-04-26T20:26:03.346736+0000","last_peered":"2024-04-26T20:26:03.346736+0000","last_clean":"2024-04-26T20:26:03.346736+0000","last_became_active":"2024-04-26T20:25:56.976895+0000","last_became_peered":"2024-04-26T20:25:56.976895+0000","last_unstale":"2024-04-26T20:26:03.346736+0000","last_undegraded":"2024-04-26T20:26:03.346736+0000","last_fullsized":"2024-04-26T20:26:03.346736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093148+0000","last_change":"2024-04-26T20:26:02.400496+0000","last_active":"2024-04-26T20:26:03.093148+0000","last_peered":"2024-04-26T20:26:03.093148+0000","last_clean":"2024-04-26T20:26:03.093148+0000","last_became_active":"2024-04-26T20:26:02.400002+0000","last_became_peered":"2024-04-26T20:26:02.400002+0000","last_unstale":"2024-04-26T20:26:03.093148+0000","last_undegraded":"2024-04-26T20:26:03.093148+0000","last_fullsized":"2024-04-26T20:26:03.093148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345843+0000","last_change":"2024-04-26T20:25:58.009127+0000","last_active":"2024-04-26T20:26:03.345843+0000","last_peered":"2024-04-26T20:26:03.345843+0000","last_clean":"2024-04-26T20:26:03.345843+0000","last_became_active":"2024-04-26T20:25:58.006542+0000","last_became_peered":"2024-04-26T20:25:58.006542+0000","last_unstale":"2024-04-26T20:26:03.345843+0000","last_undegraded":"2024-04-26T20:26:03.345843+0000","last_fullsized":"2024-04-26T20:26:03.345843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348511+0000","last_change":"2024-04-26T20:25:56.992958+0000","last_active":"2024-04-26T20:26:03.348511+0000","last_peered":"2024-04-26T20:26:03.348511+0000","last_clean":"2024-04-26T20:26:03.348511+0000","last_became_active":"2024-04-26T20:25:56.992326+0000","last_became_peered":"2024-04-26T20:25:56.992326+0000","last_unstale":"2024-04-26T20:26:03.348511+0000","last_undegraded":"2024-04-26T20:26:03.348511+0000","last_fullsized":"2024-04-26T20:26:03.348511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346294+0000","last_change":"2024-04-26T20:26:02.406897+0000","last_active":"2024-04-26T20:26:03.346294+0000","last_peered":"2024-04-26T20:26:03.346294+0000","last_clean":"2024-04-26T20:26:03.346294+0000","last_became_active":"2024-04-26T20:26:02.402553+0000","last_became_peered":"2024-04-26T20:26:02.402553+0000","last_unstale":"2024-04-26T20:26:03.346294+0000","last_undegraded":"2024-04-26T20:26:03.346294+0000","last_fullsized":"2024-04-26T20:26:03.346294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347458+0000","last_change":"2024-04-26T20:25:58.003889+0000","last_active":"2024-04-26T20:26:03.347458+0000","last_peered":"2024-04-26T20:26:03.347458+0000","last_clean":"2024-04-26T20:26:03.347458+0000","last_became_active":"2024-04-26T20:25:58.003578+0000","last_became_peered":"2024-04-26T20:25:58.003578+0000","last_unstale":"2024-04-26T20:26:03.347458+0000","last_undegraded":"2024-04-26T20:26:03.347458+0000","last_fullsized":"2024-04-26T20:26:03.347458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346054+0000","last_change":"2024-04-26T20:25:56.975939+0000","last_active":"2024-04-26T20:26:03.346054+0000","last_peered":"2024-04-26T20:26:03.346054+0000","last_clean":"2024-04-26T20:26:03.346054+0000","last_became_active":"2024-04-26T20:25:56.975669+0000","last_became_peered":"2024-04-26T20:25:56.975669+0000","last_unstale":"2024-04-26T20:26:03.346054+0000","last_undegraded":"2024-04-26T20:26:03.346054+0000","last_fullsized":"2024-04-26T20:26:03.346054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344705+0000","last_change":"2024-04-26T20:26:02.079073+0000","last_active":"2024-04-26T20:26:03.344705+0000","last_peered":"2024-04-26T20:26:03.344705+0000","last_clean":"2024-04-26T20:26:03.344705+0000","last_became_active":"2024-04-26T20:26:02.074779+0000","last_became_peered":"2024-04-26T20:26:02.074779+0000","last_unstale":"2024-04-26T20:26:03.344705+0000","last_undegraded":"2024-04-26T20:26:03.344705+0000","last_fullsized":"2024-04-26T20:26:03.344705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051836+0000","last_change":"2024-04-26T20:25:57.988308+0000","last_active":"2024-04-26T20:26:03.051836+0000","last_peered":"2024-04-26T20:26:03.051836+0000","last_clean":"2024-04-26T20:26:03.051836+0000","last_became_active":"2024-04-26T20:25:57.984254+0000","last_became_peered":"2024-04-26T20:25:57.984254+0000","last_unstale":"2024-04-26T20:26:03.051836+0000","last_undegraded":"2024-04-26T20:26:03.051836+0000","last_fullsized":"2024-04-26T20:26:03.051836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:26.721241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348724+0000","last_change":"2024-04-26T20:25:56.980841+0000","last_active":"2024-04-26T20:26:03.348724+0000","last_peered":"2024-04-26T20:26:03.348724+0000","last_clean":"2024-04-26T20:26:03.348724+0000","last_became_active":"2024-04-26T20:25:56.980215+0000","last_became_peered":"2024-04-26T20:25:56.980215+0000","last_unstale":"2024-04-26T20:26:03.348724+0000","last_undegraded":"2024-04-26T20:26:03.348724+0000","last_fullsized":"2024-04-26T20:26:03.348724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345720+0000","last_change":"2024-04-26T20:26:02.078478+0000","last_active":"2024-04-26T20:26:03.345720+0000","last_peered":"2024-04-26T20:26:03.345720+0000","last_clean":"2024-04-26T20:26:03.345720+0000","last_became_active":"2024-04-26T20:26:02.078121+0000","last_became_peered":"2024-04-26T20:26:02.078121+0000","last_unstale":"2024-04-26T20:26:03.345720+0000","last_undegraded":"2024-04-26T20:26:03.345720+0000","last_fullsized":"2024-04-26T20:26:03.345720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343920+0000","last_change":"2024-04-26T20:25:58.001121+0000","last_active":"2024-04-26T20:26:03.343920+0000","last_peered":"2024-04-26T20:26:03.343920+0000","last_clean":"2024-04-26T20:26:03.343920+0000","last_became_active":"2024-04-26T20:25:58.000412+0000","last_became_peered":"2024-04-26T20:25:58.000412+0000","last_unstale":"2024-04-26T20:26:03.343920+0000","last_undegraded":"2024-04-26T20:26:03.343920+0000","last_fullsized":"2024-04-26T20:26:03.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048086+0000","last_change":"2024-04-26T20:25:56.972300+0000","last_active":"2024-04-26T20:26:03.048086+0000","last_peered":"2024-04-26T20:26:03.048086+0000","last_clean":"2024-04-26T20:26:03.048086+0000","last_became_active":"2024-04-26T20:25:56.972020+0000","last_became_peered":"2024-04-26T20:25:56.972020+0000","last_unstale":"2024-04-26T20:26:03.048086+0000","last_undegraded":"2024-04-26T20:26:03.048086+0000","last_fullsized":"2024-04-26T20:26:03.048086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347521+0000","last_change":"2024-04-26T20:26:02.077923+0000","last_active":"2024-04-26T20:26:03.347521+0000","last_peered":"2024-04-26T20:26:03.347521+0000","last_clean":"2024-04-26T20:26:03.347521+0000","last_became_active":"2024-04-26T20:26:02.077483+0000","last_became_peered":"2024-04-26T20:26:02.077483+0000","last_unstale":"2024-04-26T20:26:03.347521+0000","last_undegraded":"2024-04-26T20:26:03.347521+0000","last_fullsized":"2024-04-26T20:26:03.347521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344678+0000","last_change":"2024-04-26T20:25:57.984633+0000","last_active":"2024-04-26T20:26:03.344678+0000","last_peered":"2024-04-26T20:26:03.344678+0000","last_clean":"2024-04-26T20:26:03.344678+0000","last_became_active":"2024-04-26T20:25:57.984262+0000","last_became_peered":"2024-04-26T20:25:57.984262+0000","last_unstale":"2024-04-26T20:26:03.344678+0000","last_undegraded":"2024-04-26T20:26:03.344678+0000","last_fullsized":"2024-04-26T20:26:03.344678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:03.002879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345671+0000","last_change":"2024-04-26T20:25:56.989467+0000","last_active":"2024-04-26T20:26:03.345671+0000","last_peered":"2024-04-26T20:26:03.345671+0000","last_clean":"2024-04-26T20:26:03.345671+0000","last_became_active":"2024-04-26T20:25:56.989183+0000","last_became_peered":"2024-04-26T20:25:56.989183+0000","last_unstale":"2024-04-26T20:26:03.345671+0000","last_undegraded":"2024-04-26T20:26:03.345671+0000","last_fullsized":"2024-04-26T20:26:03.345671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:11.650840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346929+0000","last_change":"2024-04-26T20:26:02.414455+0000","last_active":"2024-04-26T20:26:03.346929+0000","last_peered":"2024-04-26T20:26:03.346929+0000","last_clean":"2024-04-26T20:26:03.346929+0000","last_became_active":"2024-04-26T20:26:02.414064+0000","last_became_peered":"2024-04-26T20:26:02.414064+0000","last_unstale":"2024-04-26T20:26:03.346929+0000","last_undegraded":"2024-04-26T20:26:03.346929+0000","last_fullsized":"2024-04-26T20:26:03.346929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093496+0000","last_change":"2024-04-26T20:25:58.008847+0000","last_active":"2024-04-26T20:26:03.093496+0000","last_peered":"2024-04-26T20:26:03.093496+0000","last_clean":"2024-04-26T20:26:03.093496+0000","last_became_active":"2024-04-26T20:25:58.008549+0000","last_became_peered":"2024-04-26T20:25:58.008549+0000","last_unstale":"2024-04-26T20:26:03.093496+0000","last_undegraded":"2024-04-26T20:26:03.093496+0000","last_fullsized":"2024-04-26T20:26:03.093496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345379+0000","last_change":"2024-04-26T20:25:56.981771+0000","last_active":"2024-04-26T20:26:03.345379+0000","last_peered":"2024-04-26T20:26:03.345379+0000","last_clean":"2024-04-26T20:26:03.345379+0000","last_became_active":"2024-04-26T20:25:56.981406+0000","last_became_peered":"2024-04-26T20:25:56.981406+0000","last_unstale":"2024-04-26T20:26:03.345379+0000","last_undegraded":"2024-04-26T20:26:03.345379+0000","last_fullsized":"2024-04-26T20:26:03.345379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345981+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:26:03.345981+0000","last_peered":"2024-04-26T20:26:03.345981+0000","last_clean":"2024-04-26T20:26:03.345981+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:26:03.345981+0000","last_undegraded":"2024-04-26T20:26:03.345981+0000","last_fullsized":"2024-04-26T20:26:03.345981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351315+0000","last_change":"2024-04-26T20:26:02.407562+0000","last_active":"2024-04-26T20:26:03.351315+0000","last_peered":"2024-04-26T20:26:03.351315+0000","last_clean":"2024-04-26T20:26:03.351315+0000","last_became_active":"2024-04-26T20:26:02.406288+0000","last_became_peered":"2024-04-26T20:26:02.406288+0000","last_unstale":"2024-04-26T20:26:03.351315+0000","last_undegraded":"2024-04-26T20:26:03.351315+0000","last_fullsized":"2024-04-26T20:26:03.351315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347732+0000","last_change":"2024-04-26T20:25:58.014416+0000","last_active":"2024-04-26T20:26:03.347732+0000","last_peered":"2024-04-26T20:26:03.347732+0000","last_clean":"2024-04-26T20:26:03.347732+0000","last_became_active":"2024-04-26T20:25:58.014128+0000","last_became_peered":"2024-04-26T20:25:58.014128+0000","last_unstale":"2024-04-26T20:26:03.347732+0000","last_undegraded":"2024-04-26T20:26:03.347732+0000","last_fullsized":"2024-04-26T20:26:03.347732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:22.089301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048942+0000","last_change":"2024-04-26T20:25:56.974272+0000","last_active":"2024-04-26T20:26:03.048942+0000","last_peered":"2024-04-26T20:26:03.048942+0000","last_clean":"2024-04-26T20:26:03.048942+0000","last_became_active":"2024-04-26T20:25:56.972999+0000","last_became_peered":"2024-04-26T20:25:56.972999+0000","last_unstale":"2024-04-26T20:26:03.048942+0000","last_undegraded":"2024-04-26T20:26:03.048942+0000","last_fullsized":"2024-04-26T20:26:03.048942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349430+0000","last_change":"2024-04-26T20:26:02.402839+0000","last_active":"2024-04-26T20:26:03.349430+0000","last_peered":"2024-04-26T20:26:03.349430+0000","last_clean":"2024-04-26T20:26:03.349430+0000","last_became_active":"2024-04-26T20:26:02.401630+0000","last_became_peered":"2024-04-26T20:26:02.401630+0000","last_unstale":"2024-04-26T20:26:03.349430+0000","last_undegraded":"2024-04-26T20:26:03.349430+0000","last_fullsized":"2024-04-26T20:26:03.349430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346830+0000","last_change":"2024-04-26T20:25:58.009012+0000","last_active":"2024-04-26T20:26:03.346830+0000","last_peered":"2024-04-26T20:26:03.346830+0000","last_clean":"2024-04-26T20:26:03.346830+0000","last_became_active":"2024-04-26T20:25:58.008594+0000","last_became_peered":"2024-04-26T20:25:58.008594+0000","last_unstale":"2024-04-26T20:26:03.346830+0000","last_undegraded":"2024-04-26T20:26:03.346830+0000","last_fullsized":"2024-04-26T20:26:03.346830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344764+0000","last_change":"2024-04-26T20:25:56.972043+0000","last_active":"2024-04-26T20:26:03.344764+0000","last_peered":"2024-04-26T20:26:03.344764+0000","last_clean":"2024-04-26T20:26:03.344764+0000","last_became_active":"2024-04-26T20:25:56.971671+0000","last_became_peered":"2024-04-26T20:25:56.971671+0000","last_unstale":"2024-04-26T20:26:03.344764+0000","last_undegraded":"2024-04-26T20:26:03.344764+0000","last_fullsized":"2024-04-26T20:26:03.344764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346513+0000","last_change":"2024-04-26T20:26:02.073728+0000","last_active":"2024-04-26T20:26:03.346513+0000","last_peered":"2024-04-26T20:26:03.346513+0000","last_clean":"2024-04-26T20:26:03.346513+0000","last_became_active":"2024-04-26T20:26:02.073238+0000","last_became_peered":"2024-04-26T20:26:02.073238+0000","last_unstale":"2024-04-26T20:26:03.346513+0000","last_undegraded":"2024-04-26T20:26:03.346513+0000","last_fullsized":"2024-04-26T20:26:03.346513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350204+0000","last_change":"2024-04-26T20:25:58.011539+0000","last_active":"2024-04-26T20:26:03.350204+0000","last_peered":"2024-04-26T20:26:03.350204+0000","last_clean":"2024-04-26T20:26:03.350204+0000","last_became_active":"2024-04-26T20:25:58.011256+0000","last_became_peered":"2024-04-26T20:25:58.011256+0000","last_unstale":"2024-04-26T20:26:03.350204+0000","last_undegraded":"2024-04-26T20:26:03.350204+0000","last_fullsized":"2024-04-26T20:26:03.350204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:02.292377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048755+0000","last_change":"2024-04-26T20:25:56.972332+0000","last_active":"2024-04-26T20:26:03.048755+0000","last_peered":"2024-04-26T20:26:03.048755+0000","last_clean":"2024-04-26T20:26:03.048755+0000","last_became_active":"2024-04-26T20:25:56.972075+0000","last_became_peered":"2024-04-26T20:25:56.972075+0000","last_unstale":"2024-04-26T20:26:03.048755+0000","last_undegraded":"2024-04-26T20:26:03.048755+0000","last_fullsized":"2024-04-26T20:26:03.048755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346282+0000","last_change":"2024-04-26T20:26:02.414970+0000","last_active":"2024-04-26T20:26:03.346282+0000","last_peered":"2024-04-26T20:26:03.346282+0000","last_clean":"2024-04-26T20:26:03.346282+0000","last_became_active":"2024-04-26T20:26:02.414093+0000","last_became_peered":"2024-04-26T20:26:02.414093+0000","last_unstale":"2024-04-26T20:26:03.346282+0000","last_undegraded":"2024-04-26T20:26:03.346282+0000","last_fullsized":"2024-04-26T20:26:03.346282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348952+0000","last_change":"2024-04-26T20:25:58.014604+0000","last_active":"2024-04-26T20:26:03.348952+0000","last_peered":"2024-04-26T20:26:03.348952+0000","last_clean":"2024-04-26T20:26:03.348952+0000","last_became_active":"2024-04-26T20:25:58.014239+0000","last_became_peered":"2024-04-26T20:25:58.014239+0000","last_unstale":"2024-04-26T20:26:03.348952+0000","last_undegraded":"2024-04-26T20:26:03.348952+0000","last_fullsized":"2024-04-26T20:26:03.348952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:24.555699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050237+0000","last_change":"2024-04-26T20:25:56.970496+0000","last_active":"2024-04-26T20:26:03.050237+0000","last_peered":"2024-04-26T20:26:03.050237+0000","last_clean":"2024-04-26T20:26:03.050237+0000","last_became_active":"2024-04-26T20:25:56.970174+0000","last_became_peered":"2024-04-26T20:25:56.970174+0000","last_unstale":"2024-04-26T20:26:03.050237+0000","last_undegraded":"2024-04-26T20:26:03.050237+0000","last_fullsized":"2024-04-26T20:26:03.050237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:25.599135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049091+0000","last_change":"2024-04-26T20:26:02.082060+0000","last_active":"2024-04-26T20:26:03.049091+0000","last_peered":"2024-04-26T20:26:03.049091+0000","last_clean":"2024-04-26T20:26:03.049091+0000","last_became_active":"2024-04-26T20:26:02.081533+0000","last_became_peered":"2024-04-26T20:26:02.081533+0000","last_unstale":"2024-04-26T20:26:03.049091+0000","last_undegraded":"2024-04-26T20:26:03.049091+0000","last_fullsized":"2024-04-26T20:26:03.049091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:02.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346324+0000","last_change":"2024-04-26T20:25:58.008801+0000","last_active":"2024-04-26T20:26:03.346324+0000","last_peered":"2024-04-26T20:26:03.346324+0000","last_clean":"2024-04-26T20:26:03.346324+0000","last_became_active":"2024-04-26T20:25:58.008479+0000","last_became_peered":"2024-04-26T20:25:58.008479+0000","last_unstale":"2024-04-26T20:26:03.346324+0000","last_undegraded":"2024-04-26T20:26:03.346324+0000","last_fullsized":"2024-04-26T20:26:03.346324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047755+0000","last_change":"2024-04-26T20:25:56.968431+0000","last_active":"2024-04-26T20:26:03.047755+0000","last_peered":"2024-04-26T20:26:03.047755+0000","last_clean":"2024-04-26T20:26:03.047755+0000","last_became_active":"2024-04-26T20:25:56.967495+0000","last_became_peered":"2024-04-26T20:25:56.967495+0000","last_unstale":"2024-04-26T20:26:03.047755+0000","last_undegraded":"2024-04-26T20:26:03.047755+0000","last_fullsized":"2024-04-26T20:26:03.047755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346734+0000","last_change":"2024-04-26T20:26:02.078801+0000","last_active":"2024-04-26T20:26:03.346734+0000","last_peered":"2024-04-26T20:26:03.346734+0000","last_clean":"2024-04-26T20:26:03.346734+0000","last_became_active":"2024-04-26T20:26:02.078343+0000","last_became_peered":"2024-04-26T20:26:02.078343+0000","last_unstale":"2024-04-26T20:26:03.346734+0000","last_undegraded":"2024-04-26T20:26:03.346734+0000","last_fullsized":"2024-04-26T20:26:03.346734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350880+0000","last_change":"2024-04-26T20:25:57.996180+0000","last_active":"2024-04-26T20:26:03.350880+0000","last_peered":"2024-04-26T20:26:03.350880+0000","last_clean":"2024-04-26T20:26:03.350880+0000","last_became_active":"2024-04-26T20:25:57.990792+0000","last_became_peered":"2024-04-26T20:25:57.990792+0000","last_unstale":"2024-04-26T20:26:03.350880+0000","last_undegraded":"2024-04-26T20:26:03.350880+0000","last_fullsized":"2024-04-26T20:26:03.350880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349887+0000","last_change":"2024-04-26T20:25:56.974115+0000","last_active":"2024-04-26T20:26:03.349887+0000","last_peered":"2024-04-26T20:26:03.349887+0000","last_clean":"2024-04-26T20:26:03.349887+0000","last_became_active":"2024-04-26T20:25:56.973801+0000","last_became_peered":"2024-04-26T20:25:56.973801+0000","last_unstale":"2024-04-26T20:26:03.349887+0000","last_undegraded":"2024-04-26T20:26:03.349887+0000","last_fullsized":"2024-04-26T20:26:03.349887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095178+0000","last_change":"2024-04-26T20:26:02.413590+0000","last_active":"2024-04-26T20:26:03.095178+0000","last_peered":"2024-04-26T20:26:03.095178+0000","last_clean":"2024-04-26T20:26:03.095178+0000","last_became_active":"2024-04-26T20:26:02.413104+0000","last_became_peered":"2024-04-26T20:26:02.413104+0000","last_unstale":"2024-04-26T20:26:03.095178+0000","last_undegraded":"2024-04-26T20:26:03.095178+0000","last_fullsized":"2024-04-26T20:26:03.095178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:35.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347726+0000","last_change":"2024-04-26T20:25:58.014095+0000","last_active":"2024-04-26T20:26:03.347726+0000","last_peered":"2024-04-26T20:26:03.347726+0000","last_clean":"2024-04-26T20:26:03.347726+0000","last_became_active":"2024-04-26T20:25:58.012813+0000","last_became_peered":"2024-04-26T20:25:58.012813+0000","last_unstale":"2024-04-26T20:26:03.347726+0000","last_undegraded":"2024-04-26T20:26:03.347726+0000","last_fullsized":"2024-04-26T20:26:03.347726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348384+0000","last_change":"2024-04-26T20:25:56.987103+0000","last_active":"2024-04-26T20:26:03.348384+0000","last_peered":"2024-04-26T20:26:03.348384+0000","last_clean":"2024-04-26T20:26:03.348384+0000","last_became_active":"2024-04-26T20:25:56.986766+0000","last_became_peered":"2024-04-26T20:25:56.986766+0000","last_unstale":"2024-04-26T20:26:03.348384+0000","last_undegraded":"2024-04-26T20:26:03.348384+0000","last_fullsized":"2024-04-26T20:26:03.348384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094102+0000","last_change":"2024-04-26T20:26:02.400999+0000","last_active":"2024-04-26T20:26:03.094102+0000","last_peered":"2024-04-26T20:26:03.094102+0000","last_clean":"2024-04-26T20:26:03.094102+0000","last_became_active":"2024-04-26T20:26:02.399994+0000","last_became_peered":"2024-04-26T20:26:02.399994+0000","last_unstale":"2024-04-26T20:26:03.094102+0000","last_undegraded":"2024-04-26T20:26:03.094102+0000","last_fullsized":"2024-04-26T20:26:03.094102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:05.311868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345529+0000","last_change":"2024-04-26T20:25:57.996909+0000","last_active":"2024-04-26T20:26:03.345529+0000","last_peered":"2024-04-26T20:26:03.345529+0000","last_clean":"2024-04-26T20:26:03.345529+0000","last_became_active":"2024-04-26T20:25:57.996635+0000","last_became_peered":"2024-04-26T20:25:57.996635+0000","last_unstale":"2024-04-26T20:26:03.345529+0000","last_undegraded":"2024-04-26T20:26:03.345529+0000","last_fullsized":"2024-04-26T20:26:03.345529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346908+0000","last_change":"2024-04-26T20:25:56.974019+0000","last_active":"2024-04-26T20:26:03.346908+0000","last_peered":"2024-04-26T20:26:03.346908+0000","last_clean":"2024-04-26T20:26:03.346908+0000","last_became_active":"2024-04-26T20:25:56.971170+0000","last_became_peered":"2024-04-26T20:25:56.971170+0000","last_unstale":"2024-04-26T20:26:03.346908+0000","last_undegraded":"2024-04-26T20:26:03.346908+0000","last_fullsized":"2024-04-26T20:26:03.346908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347253+0000","last_change":"2024-04-26T20:26:02.402012+0000","last_active":"2024-04-26T20:26:03.347253+0000","last_peered":"2024-04-26T20:26:03.347253+0000","last_clean":"2024-04-26T20:26:03.347253+0000","last_became_active":"2024-04-26T20:26:02.400605+0000","last_became_peered":"2024-04-26T20:26:02.400605+0000","last_unstale":"2024-04-26T20:26:03.347253+0000","last_undegraded":"2024-04-26T20:26:03.347253+0000","last_fullsized":"2024-04-26T20:26:03.347253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347168+0000","last_change":"2024-04-26T20:25:58.001764+0000","last_active":"2024-04-26T20:26:03.347168+0000","last_peered":"2024-04-26T20:26:03.347168+0000","last_clean":"2024-04-26T20:26:03.347168+0000","last_became_active":"2024-04-26T20:25:58.001502+0000","last_became_peered":"2024-04-26T20:25:58.001502+0000","last_unstale":"2024-04-26T20:26:03.347168+0000","last_undegraded":"2024-04-26T20:26:03.347168+0000","last_fullsized":"2024-04-26T20:26:03.347168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:16.030394+0000","last_change":"2024-04-26T20:25:56.984259+0000","last_active":"2024-04-26T20:26:16.030394+0000","last_peered":"2024-04-26T20:26:16.030394+0000","last_clean":"2024-04-26T20:26:16.030394+0000","last_became_active":"2024-04-26T20:25:56.983974+0000","last_became_peered":"2024-04-26T20:25:56.983974+0000","last_unstale":"2024-04-26T20:26:16.030394+0000","last_undegraded":"2024-04-26T20:26:16.030394+0000","last_fullsized":"2024-04-26T20:26:16.030394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345142+0000","last_change":"2024-04-26T20:26:02.080802+0000","last_active":"2024-04-26T20:26:03.345142+0000","last_peered":"2024-04-26T20:26:03.345142+0000","last_clean":"2024-04-26T20:26:03.345142+0000","last_became_active":"2024-04-26T20:26:02.079790+0000","last_became_peered":"2024-04-26T20:26:02.079790+0000","last_unstale":"2024-04-26T20:26:03.345142+0000","last_undegraded":"2024-04-26T20:26:03.345142+0000","last_fullsized":"2024-04-26T20:26:03.345142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050516+0000","last_change":"2024-04-26T20:25:57.999695+0000","last_active":"2024-04-26T20:26:03.050516+0000","last_peered":"2024-04-26T20:26:03.050516+0000","last_clean":"2024-04-26T20:26:03.050516+0000","last_became_active":"2024-04-26T20:25:57.999476+0000","last_became_peered":"2024-04-26T20:25:57.999476+0000","last_unstale":"2024-04-26T20:26:03.050516+0000","last_undegraded":"2024-04-26T20:26:03.050516+0000","last_fullsized":"2024-04-26T20:26:03.050516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344567+0000","last_change":"2024-04-26T20:25:56.984463+0000","last_active":"2024-04-26T20:26:03.344567+0000","last_peered":"2024-04-26T20:26:03.344567+0000","last_clean":"2024-04-26T20:26:03.344567+0000","last_became_active":"2024-04-26T20:25:56.984176+0000","last_became_peered":"2024-04-26T20:25:56.984176+0000","last_unstale":"2024-04-26T20:26:03.344567+0000","last_undegraded":"2024-04-26T20:26:03.344567+0000","last_fullsized":"2024-04-26T20:26:03.344567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051617+0000","last_change":"2024-04-26T20:26:02.079338+0000","last_active":"2024-04-26T20:26:03.051617+0000","last_peered":"2024-04-26T20:26:03.051617+0000","last_clean":"2024-04-26T20:26:03.051617+0000","last_became_active":"2024-04-26T20:26:02.078830+0000","last_became_peered":"2024-04-26T20:26:02.078830+0000","last_unstale":"2024-04-26T20:26:03.051617+0000","last_undegraded":"2024-04-26T20:26:03.051617+0000","last_fullsized":"2024-04-26T20:26:03.051617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347553+0000","last_change":"2024-04-26T20:25:58.008157+0000","last_active":"2024-04-26T20:26:03.347553+0000","last_peered":"2024-04-26T20:26:03.347553+0000","last_clean":"2024-04-26T20:26:03.347553+0000","last_became_active":"2024-04-26T20:25:58.005542+0000","last_became_peered":"2024-04-26T20:25:58.005542+0000","last_unstale":"2024-04-26T20:26:03.347553+0000","last_undegraded":"2024-04-26T20:26:03.347553+0000","last_fullsized":"2024-04-26T20:26:03.347553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348327+0000","last_change":"2024-04-26T20:25:56.987547+0000","last_active":"2024-04-26T20:26:03.348327+0000","last_peered":"2024-04-26T20:26:03.348327+0000","last_clean":"2024-04-26T20:26:03.348327+0000","last_became_active":"2024-04-26T20:25:56.985375+0000","last_became_peered":"2024-04-26T20:25:56.985375+0000","last_unstale":"2024-04-26T20:26:03.348327+0000","last_undegraded":"2024-04-26T20:26:03.348327+0000","last_fullsized":"2024-04-26T20:26:03.348327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093045+0000","last_change":"2024-04-26T20:26:02.073521+0000","last_active":"2024-04-26T20:26:03.093045+0000","last_peered":"2024-04-26T20:26:03.093045+0000","last_clean":"2024-04-26T20:26:03.093045+0000","last_became_active":"2024-04-26T20:26:02.073134+0000","last_became_peered":"2024-04-26T20:26:02.073134+0000","last_unstale":"2024-04-26T20:26:03.093045+0000","last_undegraded":"2024-04-26T20:26:03.093045+0000","last_fullsized":"2024-04-26T20:26:03.093045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:06.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348491+0000","last_change":"2024-04-26T20:25:58.011178+0000","last_active":"2024-04-26T20:26:03.348491+0000","last_peered":"2024-04-26T20:26:03.348491+0000","last_clean":"2024-04-26T20:26:03.348491+0000","last_became_active":"2024-04-26T20:25:58.010833+0000","last_became_peered":"2024-04-26T20:25:58.010833+0000","last_unstale":"2024-04-26T20:26:03.348491+0000","last_undegraded":"2024-04-26T20:26:03.348491+0000","last_fullsized":"2024-04-26T20:26:03.348491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351644+0000","last_change":"2024-04-26T20:25:56.986174+0000","last_active":"2024-04-26T20:26:03.351644+0000","last_peered":"2024-04-26T20:26:03.351644+0000","last_clean":"2024-04-26T20:26:03.351644+0000","last_became_active":"2024-04-26T20:25:56.985869+0000","last_became_peered":"2024-04-26T20:25:56.985869+0000","last_unstale":"2024-04-26T20:26:03.351644+0000","last_undegraded":"2024-04-26T20:26:03.351644+0000","last_fullsized":"2024-04-26T20:26:03.351644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093097+0000","last_change":"2024-04-26T20:26:02.413559+0000","last_active":"2024-04-26T20:26:03.093097+0000","last_peered":"2024-04-26T20:26:03.093097+0000","last_clean":"2024-04-26T20:26:03.093097+0000","last_became_active":"2024-04-26T20:26:02.413096+0000","last_became_peered":"2024-04-26T20:26:02.413096+0000","last_unstale":"2024-04-26T20:26:03.093097+0000","last_undegraded":"2024-04-26T20:26:03.093097+0000","last_fullsized":"2024-04-26T20:26:03.093097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:44.245008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346456+0000","last_change":"2024-04-26T20:26:02.413921+0000","last_active":"2024-04-26T20:26:03.346456+0000","last_peered":"2024-04-26T20:26:03.346456+0000","last_clean":"2024-04-26T20:26:03.346456+0000","last_became_active":"2024-04-26T20:26:02.413388+0000","last_became_peered":"2024-04-26T20:26:02.413388+0000","last_unstale":"2024-04-26T20:26:03.346456+0000","last_undegraded":"2024-04-26T20:26:03.346456+0000","last_fullsized":"2024-04-26T20:26:03.346456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345276+0000","last_change":"2024-04-26T20:25:58.004583+0000","last_active":"2024-04-26T20:26:03.345276+0000","last_peered":"2024-04-26T20:26:03.345276+0000","last_clean":"2024-04-26T20:26:03.345276+0000","last_became_active":"2024-04-26T20:25:58.004228+0000","last_became_peered":"2024-04-26T20:25:58.004228+0000","last_unstale":"2024-04-26T20:26:03.345276+0000","last_undegraded":"2024-04-26T20:26:03.345276+0000","last_fullsized":"2024-04-26T20:26:03.345276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346675+0000","last_change":"2024-04-26T20:25:56.985910+0000","last_active":"2024-04-26T20:26:03.346675+0000","last_peered":"2024-04-26T20:26:03.346675+0000","last_clean":"2024-04-26T20:26:03.346675+0000","last_became_active":"2024-04-26T20:25:56.985636+0000","last_became_peered":"2024-04-26T20:25:56.985636+0000","last_unstale":"2024-04-26T20:26:03.346675+0000","last_undegraded":"2024-04-26T20:26:03.346675+0000","last_fullsized":"2024-04-26T20:26:03.346675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351558+0000","last_change":"2024-04-26T20:26:02.079470+0000","last_active":"2024-04-26T20:26:03.351558+0000","last_peered":"2024-04-26T20:26:03.351558+0000","last_clean":"2024-04-26T20:26:03.351558+0000","last_became_active":"2024-04-26T20:26:02.078853+0000","last_became_peered":"2024-04-26T20:26:02.078853+0000","last_unstale":"2024-04-26T20:26:03.351558+0000","last_undegraded":"2024-04-26T20:26:03.351558+0000","last_fullsized":"2024-04-26T20:26:03.351558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093119+0000","last_change":"2024-04-26T20:25:58.006907+0000","last_active":"2024-04-26T20:26:03.093119+0000","last_peered":"2024-04-26T20:26:03.093119+0000","last_clean":"2024-04-26T20:26:03.093119+0000","last_became_active":"2024-04-26T20:25:58.006370+0000","last_became_peered":"2024-04-26T20:25:58.006370+0000","last_unstale":"2024-04-26T20:26:03.093119+0000","last_undegraded":"2024-04-26T20:26:03.093119+0000","last_fullsized":"2024-04-26T20:26:03.093119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093249+0000","last_change":"2024-04-26T20:25:56.982952+0000","last_active":"2024-04-26T20:26:03.093249+0000","last_peered":"2024-04-26T20:26:03.093249+0000","last_clean":"2024-04-26T20:26:03.093249+0000","last_became_active":"2024-04-26T20:25:56.982240+0000","last_became_peered":"2024-04-26T20:25:56.982240+0000","last_unstale":"2024-04-26T20:26:03.093249+0000","last_undegraded":"2024-04-26T20:26:03.093249+0000","last_fullsized":"2024-04-26T20:26:03.093249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349655+0000","last_change":"2024-04-26T20:25:56.982228+0000","last_active":"2024-04-26T20:26:03.349655+0000","last_peered":"2024-04-26T20:26:03.349655+0000","last_clean":"2024-04-26T20:26:03.349655+0000","last_became_active":"2024-04-26T20:25:56.981083+0000","last_became_peered":"2024-04-26T20:25:56.981083+0000","last_unstale":"2024-04-26T20:26:03.349655+0000","last_undegraded":"2024-04-26T20:26:03.349655+0000","last_fullsized":"2024-04-26T20:26:03.349655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346695+0000","last_change":"2024-04-26T20:25:58.001685+0000","last_active":"2024-04-26T20:26:03.346695+0000","last_peered":"2024-04-26T20:26:03.346695+0000","last_clean":"2024-04-26T20:26:03.346695+0000","last_became_active":"2024-04-26T20:25:58.001160+0000","last_became_peered":"2024-04-26T20:25:58.001160+0000","last_unstale":"2024-04-26T20:26:03.346695+0000","last_undegraded":"2024-04-26T20:26:03.346695+0000","last_fullsized":"2024-04-26T20:26:03.346695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048776+0000","last_change":"2024-04-26T20:26:02.078797+0000","last_active":"2024-04-26T20:26:03.048776+0000","last_peered":"2024-04-26T20:26:03.048776+0000","last_clean":"2024-04-26T20:26:03.048776+0000","last_became_active":"2024-04-26T20:26:02.078444+0000","last_became_peered":"2024-04-26T20:26:02.078444+0000","last_unstale":"2024-04-26T20:26:03.048776+0000","last_undegraded":"2024-04-26T20:26:03.048776+0000","last_fullsized":"2024-04-26T20:26:03.048776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:16.872808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093946+0000","last_change":"2024-04-26T20:25:56.986739+0000","last_active":"2024-04-26T20:26:03.093946+0000","last_peered":"2024-04-26T20:26:03.093946+0000","last_clean":"2024-04-26T20:26:03.093946+0000","last_became_active":"2024-04-26T20:25:56.986399+0000","last_became_peered":"2024-04-26T20:25:56.986399+0000","last_unstale":"2024-04-26T20:26:03.093946+0000","last_undegraded":"2024-04-26T20:26:03.093946+0000","last_fullsized":"2024-04-26T20:26:03.093946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050966+0000","last_change":"2024-04-26T20:25:57.999363+0000","last_active":"2024-04-26T20:26:03.050966+0000","last_peered":"2024-04-26T20:26:03.050966+0000","last_clean":"2024-04-26T20:26:03.050966+0000","last_became_active":"2024-04-26T20:25:57.995297+0000","last_became_peered":"2024-04-26T20:25:57.995297+0000","last_unstale":"2024-04-26T20:26:03.050966+0000","last_undegraded":"2024-04-26T20:26:03.050966+0000","last_fullsized":"2024-04-26T20:26:03.050966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.671294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094404+0000","last_change":"2024-04-26T20:26:02.075522+0000","last_active":"2024-04-26T20:26:03.094404+0000","last_peered":"2024-04-26T20:26:03.094404+0000","last_clean":"2024-04-26T20:26:03.094404+0000","last_became_active":"2024-04-26T20:26:02.075089+0000","last_became_peered":"2024-04-26T20:26:02.075089+0000","last_unstale":"2024-04-26T20:26:03.094404+0000","last_undegraded":"2024-04-26T20:26:03.094404+0000","last_fullsized":"2024-04-26T20:26:03.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:23.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350468+0000","last_change":"2024-04-26T20:25:57.990722+0000","last_active":"2024-04-26T20:26:03.350468+0000","last_peered":"2024-04-26T20:26:03.350468+0000","last_clean":"2024-04-26T20:26:03.350468+0000","last_became_active":"2024-04-26T20:25:57.990382+0000","last_became_peered":"2024-04-26T20:25:57.990382+0000","last_unstale":"2024-04-26T20:26:03.350468+0000","last_undegraded":"2024-04-26T20:26:03.350468+0000","last_fullsized":"2024-04-26T20:26:03.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047754+0000","last_change":"2024-04-26T20:25:58.014226+0000","last_active":"2024-04-26T20:26:03.047754+0000","last_peered":"2024-04-26T20:26:03.047754+0000","last_clean":"2024-04-26T20:26:03.047754+0000","last_became_active":"2024-04-26T20:25:58.013319+0000","last_became_peered":"2024-04-26T20:25:58.013319+0000","last_unstale":"2024-04-26T20:26:03.047754+0000","last_undegraded":"2024-04-26T20:26:03.047754+0000","last_fullsized":"2024-04-26T20:26:03.047754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348155+0000","last_change":"2024-04-26T20:25:58.008173+0000","last_active":"2024-04-26T20:26:03.348155+0000","last_peered":"2024-04-26T20:26:03.348155+0000","last_clean":"2024-04-26T20:26:03.348155+0000","last_became_active":"2024-04-26T20:25:58.007377+0000","last_became_peered":"2024-04-26T20:25:58.007377+0000","last_unstale":"2024-04-26T20:26:03.348155+0000","last_undegraded":"2024-04-26T20:26:03.348155+0000","last_fullsized":"2024-04-26T20:26:03.348155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346409+0000","last_change":"2024-04-26T20:25:58.003184+0000","last_active":"2024-04-26T20:26:03.346409+0000","last_peered":"2024-04-26T20:26:03.346409+0000","last_clean":"2024-04-26T20:26:03.346409+0000","last_became_active":"2024-04-26T20:25:58.002849+0000","last_became_peered":"2024-04-26T20:25:58.002849+0000","last_unstale":"2024-04-26T20:26:03.346409+0000","last_undegraded":"2024-04-26T20:26:03.346409+0000","last_fullsized":"2024-04-26T20:26:03.346409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094328+0000","last_change":"2024-04-26T20:25:57.994419+0000","last_active":"2024-04-26T20:26:03.094328+0000","last_peered":"2024-04-26T20:26:03.094328+0000","last_clean":"2024-04-26T20:26:03.094328+0000","last_became_active":"2024-04-26T20:25:57.994077+0000","last_became_peered":"2024-04-26T20:25:57.994077+0000","last_unstale":"2024-04-26T20:26:03.094328+0000","last_undegraded":"2024-04-26T20:26:03.094328+0000","last_fullsized":"2024-04-26T20:26:03.094328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344289+0000","last_change":"2024-04-26T20:25:58.000446+0000","last_active":"2024-04-26T20:26:03.344289+0000","last_peered":"2024-04-26T20:26:03.344289+0000","last_clean":"2024-04-26T20:26:03.344289+0000","last_became_active":"2024-04-26T20:25:58.000166+0000","last_became_peered":"2024-04-26T20:25:58.000166+0000","last_unstale":"2024-04-26T20:26:03.344289+0000","last_undegraded":"2024-04-26T20:26:03.344289+0000","last_fullsized":"2024-04-26T20:26:03.344289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351690+0000","last_change":"2024-04-26T20:25:57.996288+0000","last_active":"2024-04-26T20:26:03.351690+0000","last_peered":"2024-04-26T20:26:03.351690+0000","last_clean":"2024-04-26T20:26:03.351690+0000","last_became_active":"2024-04-26T20:25:57.995978+0000","last_became_peered":"2024-04-26T20:25:57.995978+0000","last_unstale":"2024-04-26T20:26:03.351690+0000","last_undegraded":"2024-04-26T20:26:03.351690+0000","last_fullsized":"2024-04-26T20:26:03.351690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094950+0000","last_change":"2024-04-26T20:25:57.993394+0000","last_active":"2024-04-26T20:26:03.094950+0000","last_peered":"2024-04-26T20:26:03.094950+0000","last_clean":"2024-04-26T20:26:03.094950+0000","last_became_active":"2024-04-26T20:25:57.993023+0000","last_became_peered":"2024-04-26T20:25:57.993023+0000","last_unstale":"2024-04-26T20:26:03.094950+0000","last_undegraded":"2024-04-26T20:26:03.094950+0000","last_fullsized":"2024-04-26T20:26:03.094950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346415+0000","last_change":"2024-04-26T20:25:57.997685+0000","last_active":"2024-04-26T20:26:03.346415+0000","last_peered":"2024-04-26T20:26:03.346415+0000","last_clean":"2024-04-26T20:26:03.346415+0000","last_became_active":"2024-04-26T20:25:57.997422+0000","last_became_peered":"2024-04-26T20:25:57.997422+0000","last_unstale":"2024-04-26T20:26:03.346415+0000","last_undegraded":"2024-04-26T20:26:03.346415+0000","last_fullsized":"2024-04-26T20:26:03.346415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093095+0000","last_change":"2024-04-26T20:25:58.003864+0000","last_active":"2024-04-26T20:26:03.093095+0000","last_peered":"2024-04-26T20:26:03.093095+0000","last_clean":"2024-04-26T20:26:03.093095+0000","last_became_active":"2024-04-26T20:25:58.003553+0000","last_became_peered":"2024-04-26T20:25:58.003553+0000","last_unstale":"2024-04-26T20:26:03.093095+0000","last_undegraded":"2024-04-26T20:26:03.093095+0000","last_fullsized":"2024-04-26T20:26:03.093095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347138+0000","last_change":"2024-04-26T20:25:58.007689+0000","last_active":"2024-04-26T20:26:03.347138+0000","last_peered":"2024-04-26T20:26:03.347138+0000","last_clean":"2024-04-26T20:26:03.347138+0000","last_became_active":"2024-04-26T20:25:58.006743+0000","last_became_peered":"2024-04-26T20:25:58.006743+0000","last_unstale":"2024-04-26T20:26:03.347138+0000","last_undegraded":"2024-04-26T20:26:03.347138+0000","last_fullsized":"2024-04-26T20:26:03.347138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346030+0000","last_change":"2024-04-26T20:25:58.001516+0000","last_active":"2024-04-26T20:26:03.346030+0000","last_peered":"2024-04-26T20:26:03.346030+0000","last_clean":"2024-04-26T20:26:03.346030+0000","last_became_active":"2024-04-26T20:25:58.001241+0000","last_became_peered":"2024-04-26T20:25:58.001241+0000","last_unstale":"2024-04-26T20:26:03.346030+0000","last_undegraded":"2024-04-26T20:26:03.346030+0000","last_fullsized":"2024-04-26T20:26:03.346030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350860+0000","last_change":"2024-04-26T20:25:57.993316+0000","last_active":"2024-04-26T20:26:03.350860+0000","last_peered":"2024-04-26T20:26:03.350860+0000","last_clean":"2024-04-26T20:26:03.350860+0000","last_became_active":"2024-04-26T20:25:57.991497+0000","last_became_peered":"2024-04-26T20:25:57.991497+0000","last_unstale":"2024-04-26T20:26:03.350860+0000","last_undegraded":"2024-04-26T20:26:03.350860+0000","last_fullsized":"2024-04-26T20:26:03.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348333+0000","last_change":"2024-04-26T20:25:58.004031+0000","last_active":"2024-04-26T20:26:03.348333+0000","last_peered":"2024-04-26T20:26:03.348333+0000","last_clean":"2024-04-26T20:26:03.348333+0000","last_became_active":"2024-04-26T20:25:58.003728+0000","last_became_peered":"2024-04-26T20:25:58.003728+0000","last_unstale":"2024-04-26T20:26:03.348333+0000","last_undegraded":"2024-04-26T20:26:03.348333+0000","last_fullsized":"2024-04-26T20:26:03.348333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:28.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048584+0000","last_change":"2024-04-26T20:25:57.986712+0000","last_active":"2024-04-26T20:26:03.048584+0000","last_peered":"2024-04-26T20:26:03.048584+0000","last_clean":"2024-04-26T20:26:03.048584+0000","last_became_active":"2024-04-26T20:25:57.986233+0000","last_became_peered":"2024-04-26T20:25:57.986233+0000","last_unstale":"2024-04-26T20:26:03.048584+0000","last_undegraded":"2024-04-26T20:26:03.048584+0000","last_fullsized":"2024-04-26T20:26:03.048584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346524+0000","last_change":"2024-04-26T20:25:58.011317+0000","last_active":"2024-04-26T20:26:03.346524+0000","last_peered":"2024-04-26T20:26:03.346524+0000","last_clean":"2024-04-26T20:26:03.346524+0000","last_became_active":"2024-04-26T20:25:58.010881+0000","last_became_peered":"2024-04-26T20:25:58.010881+0000","last_unstale":"2024-04-26T20:26:03.346524+0000","last_undegraded":"2024-04-26T20:26:03.346524+0000","last_fullsized":"2024-04-26T20:26:03.346524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344914+0000","last_change":"2024-04-26T20:25:58.018419+0000","last_active":"2024-04-26T20:26:03.344914+0000","last_peered":"2024-04-26T20:26:03.344914+0000","last_clean":"2024-04-26T20:26:03.344914+0000","last_became_active":"2024-04-26T20:25:58.018134+0000","last_became_peered":"2024-04-26T20:25:58.018134+0000","last_unstale":"2024-04-26T20:26:03.344914+0000","last_undegraded":"2024-04-26T20:26:03.344914+0000","last_fullsized":"2024-04-26T20:26:03.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343902+0000","last_change":"2024-04-26T20:25:58.008263+0000","last_active":"2024-04-26T20:26:03.343902+0000","last_peered":"2024-04-26T20:26:03.343902+0000","last_clean":"2024-04-26T20:26:03.343902+0000","last_became_active":"2024-04-26T20:25:58.007268+0000","last_became_peered":"2024-04-26T20:25:58.007268+0000","last_unstale":"2024-04-26T20:26:03.343902+0000","last_undegraded":"2024-04-26T20:26:03.343902+0000","last_fullsized":"2024-04-26T20:26:03.343902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345391+0000","last_change":"2024-04-26T20:25:58.011832+0000","last_active":"2024-04-26T20:26:03.345391+0000","last_peered":"2024-04-26T20:26:03.345391+0000","last_clean":"2024-04-26T20:26:03.345391+0000","last_became_active":"2024-04-26T20:25:58.011007+0000","last_became_peered":"2024-04-26T20:25:58.011007+0000","last_unstale":"2024-04-26T20:26:03.345391+0000","last_undegraded":"2024-04-26T20:26:03.345391+0000","last_fullsized":"2024-04-26T20:26:03.345391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:38.909585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093783+0000","last_change":"2024-04-26T20:25:57.993780+0000","last_active":"2024-04-26T20:26:03.093783+0000","last_peered":"2024-04-26T20:26:03.093783+0000","last_clean":"2024-04-26T20:26:03.093783+0000","last_became_active":"2024-04-26T20:25:57.993444+0000","last_became_peered":"2024-04-26T20:25:57.993444+0000","last_unstale":"2024-04-26T20:26:03.093783+0000","last_undegraded":"2024-04-26T20:26:03.093783+0000","last_fullsized":"2024-04-26T20:26:03.093783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050252+0000","last_change":"2024-04-26T20:25:57.985098+0000","last_active":"2024-04-26T20:26:03.050252+0000","last_peered":"2024-04-26T20:26:03.050252+0000","last_clean":"2024-04-26T20:26:03.050252+0000","last_became_active":"2024-04-26T20:25:57.984739+0000","last_became_peered":"2024-04-26T20:25:57.984739+0000","last_unstale":"2024-04-26T20:26:03.050252+0000","last_undegraded":"2024-04-26T20:26:03.050252+0000","last_fullsized":"2024-04-26T20:26:03.050252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351223+0000","last_change":"2024-04-26T20:25:58.007442+0000","last_active":"2024-04-26T20:26:03.351223+0000","last_peered":"2024-04-26T20:26:03.351223+0000","last_clean":"2024-04-26T20:26:03.351223+0000","last_became_active":"2024-04-26T20:25:58.007150+0000","last_became_peered":"2024-04-26T20:25:58.007150+0000","last_unstale":"2024-04-26T20:26:03.351223+0000","last_undegraded":"2024-04-26T20:26:03.351223+0000","last_fullsized":"2024-04-26T20:26:03.351223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351686+0000","last_change":"2024-04-26T20:25:58.007696+0000","last_active":"2024-04-26T20:26:03.351686+0000","last_peered":"2024-04-26T20:26:03.351686+0000","last_clean":"2024-04-26T20:26:03.351686+0000","last_became_active":"2024-04-26T20:25:58.007415+0000","last_became_peered":"2024-04-26T20:25:58.007415+0000","last_unstale":"2024-04-26T20:26:03.351686+0000","last_undegraded":"2024-04-26T20:26:03.351686+0000","last_fullsized":"2024-04-26T20:26:03.351686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350018+0000","last_change":"2024-04-26T20:25:58.014108+0000","last_active":"2024-04-26T20:26:03.350018+0000","last_peered":"2024-04-26T20:26:03.350018+0000","last_clean":"2024-04-26T20:26:03.350018+0000","last_became_active":"2024-04-26T20:25:58.012072+0000","last_became_peered":"2024-04-26T20:25:58.012072+0000","last_unstale":"2024-04-26T20:26:03.350018+0000","last_undegraded":"2024-04-26T20:26:03.350018+0000","last_fullsized":"2024-04-26T20:26:03.350018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:10.380322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092736+0000","last_change":"2024-04-26T20:25:58.004587+0000","last_active":"2024-04-26T20:26:03.092736+0000","last_peered":"2024-04-26T20:26:03.092736+0000","last_clean":"2024-04-26T20:26:03.092736+0000","last_became_active":"2024-04-26T20:25:58.003068+0000","last_became_peered":"2024-04-26T20:25:58.003068+0000","last_unstale":"2024-04-26T20:26:03.092736+0000","last_undegraded":"2024-04-26T20:26:03.092736+0000","last_fullsized":"2024-04-26T20:26:03.092736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345668+0000","last_change":"2024-04-26T20:25:58.017953+0000","last_active":"2024-04-26T20:26:03.345668+0000","last_peered":"2024-04-26T20:26:03.345668+0000","last_clean":"2024-04-26T20:26:03.345668+0000","last_became_active":"2024-04-26T20:25:58.017321+0000","last_became_peered":"2024-04-26T20:25:58.017321+0000","last_unstale":"2024-04-26T20:26:03.345668+0000","last_undegraded":"2024-04-26T20:26:03.345668+0000","last_fullsized":"2024-04-26T20:26:03.345668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346846+0000","last_change":"2024-04-26T20:25:58.007941+0000","last_active":"2024-04-26T20:26:03.346846+0000","last_peered":"2024-04-26T20:26:03.346846+0000","last_clean":"2024-04-26T20:26:03.346846+0000","last_became_active":"2024-04-26T20:25:58.007043+0000","last_became_peered":"2024-04-26T20:25:58.007043+0000","last_unstale":"2024-04-26T20:26:03.346846+0000","last_undegraded":"2024-04-26T20:26:03.346846+0000","last_fullsized":"2024-04-26T20:26:03.346846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347334+0000","last_change":"2024-04-26T20:25:58.003034+0000","last_active":"2024-04-26T20:26:03.347334+0000","last_peered":"2024-04-26T20:26:03.347334+0000","last_clean":"2024-04-26T20:26:03.347334+0000","last_became_active":"2024-04-26T20:25:58.002792+0000","last_became_peered":"2024-04-26T20:25:58.002792+0000","last_unstale":"2024-04-26T20:26:03.347334+0000","last_undegraded":"2024-04-26T20:26:03.347334+0000","last_fullsized":"2024-04-26T20:26:03.347334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346106+0000","last_change":"2024-04-26T20:25:58.008698+0000","last_active":"2024-04-26T20:26:03.346106+0000","last_peered":"2024-04-26T20:26:03.346106+0000","last_clean":"2024-04-26T20:26:03.346106+0000","last_became_active":"2024-04-26T20:25:58.008279+0000","last_became_peered":"2024-04-26T20:25:58.008279+0000","last_unstale":"2024-04-26T20:26:03.346106+0000","last_undegraded":"2024-04-26T20:26:03.346106+0000","last_fullsized":"2024-04-26T20:26:03.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346355+0000","last_change":"2024-04-26T20:25:57.985508+0000","last_active":"2024-04-26T20:26:03.346355+0000","last_peered":"2024-04-26T20:26:03.346355+0000","last_clean":"2024-04-26T20:26:03.346355+0000","last_became_active":"2024-04-26T20:25:57.985227+0000","last_became_peered":"2024-04-26T20:25:57.985227+0000","last_unstale":"2024-04-26T20:26:03.346355+0000","last_undegraded":"2024-04-26T20:26:03.346355+0000","last_fullsized":"2024-04-26T20:26:03.346355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346661+0000","last_change":"2024-04-26T20:25:57.989552+0000","last_active":"2024-04-26T20:26:03.346661+0000","last_peered":"2024-04-26T20:26:03.346661+0000","last_clean":"2024-04-26T20:26:03.346661+0000","last_became_active":"2024-04-26T20:25:57.989141+0000","last_became_peered":"2024-04-26T20:25:57.989141+0000","last_unstale":"2024-04-26T20:26:03.346661+0000","last_undegraded":"2024-04-26T20:26:03.346661+0000","last_fullsized":"2024-04-26T20:26:03.346661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349524+0000","last_change":"2024-04-26T20:25:58.003504+0000","last_active":"2024-04-26T20:26:03.349524+0000","last_peered":"2024-04-26T20:26:03.349524+0000","last_clean":"2024-04-26T20:26:03.349524+0000","last_became_active":"2024-04-26T20:25:58.002880+0000","last_became_peered":"2024-04-26T20:25:58.002880+0000","last_unstale":"2024-04-26T20:26:03.349524+0000","last_undegraded":"2024-04-26T20:26:03.349524+0000","last_fullsized":"2024-04-26T20:26:03.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:47.751608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:32.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:26:32.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.554+0000 7fea1a800640 1 Processor -- start 2024-04-26T20:26:32.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.555+0000 7fea1a800640 1 -- start start 2024-04-26T20:26:32.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea1a800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea140b5b80 0x7fea140d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea1a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea140d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea1a800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea140d0480 con 0x7fea140d1620 2024-04-26T20:26:32.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea1a800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fea140d0ce0 con 0x7fea140b5b80 2024-04-26T20:26:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea1a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fea140d0720 con 0x7fea140d2800 2024-04-26T20:26:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea19400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fea140d1620 0x7fea140d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41560/0 (socket says 172.21.15.43:41560) 2024-04-26T20:26:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea19400640 1 -- 172.21.15.43:0/3252670841 learned_addr learned my addr 172.21.15.43:0/3252670841 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea18a00640 1 --2- 172.21.15.43:0/3252670841 >> v2:172.21.15.43:3300/0 conn(0x7fea140b5b80 0x7fea140d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea19e00640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea140d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.556+0000 7fea13e00640 1 -- 172.21.15.43:0/3252670841 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883416105 0 0) 0x7fea140d0480 con 0x7fea140d1620 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.557+0000 7fea13e00640 1 -- 172.21.15.43:0/3252670841 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9f8003520 con 0x7fea140d1620 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.557+0000 7fea18a00640 1 -- 172.21.15.43:0/3252670841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 msgr2=0x7fea140d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.557+0000 7fea18a00640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea140d2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.557+0000 7fea18a00640 1 -- 172.21.15.43:0/3252670841 >> v1:172.21.15.43:6789/0 conn(0x7fea140d1620 legacy=0x7fea140d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.557+0000 7fea18a00640 1 -- 172.21.15.43:0/3252670841 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140d0a00 con 0x7fea140b5b80 2024-04-26T20:26:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea18a00640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140b5b80 0x7fea140d5750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea000097a0 tx=0x7fea0002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f68d3683880d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea13e00640 1 -- 172.21.15.43:0/3252670841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00037070 con 0x7fea140b5b80 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea13e00640 1 -- 172.21.15.43:0/3252670841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0002ae50 con 0x7fea140b5b80 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea13e00640 1 -- 172.21.15.43:0/3252670841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00032660 con 0x7fea140b5b80 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 -- 172.21.15.43:0/3252670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140b5b80 msgr2=0x7fea140d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140b5b80 0x7fea140d5750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea000097a0 tx=0x7fea0002a100 comp rx=0 tx=0).stop 2024-04-26T20:26:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 -- 172.21.15.43:0/3252670841 shutdown_connections 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea140d2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 --2- 172.21.15.43:0/3252670841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140b5b80 0x7fea140d5750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.558+0000 7fea1a800640 1 -- 172.21.15.43:0/3252670841 >> 172.21.15.43:0/3252670841 conn(0x7fea140d4c30 msgr2=0x7fea140d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.559+0000 7fea1a800640 1 -- 172.21.15.43:0/3252670841 shutdown_connections 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.559+0000 7fea1a800640 1 -- 172.21.15.43:0/3252670841 wait complete. 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.560+0000 7fea1a800640 1 Processor -- start 2024-04-26T20:26:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.560+0000 7fea1a800640 1 -- start start 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea140b5b80 0x7fea140cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea1415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea140d3350 con 0x7fea140d1620 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea140d31d0 con 0x7fea140b5b80 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea1a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fea140d34d0 con 0x7fea140d2800 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea18a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea18a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54920/0 (socket says 172.21.15.43:54920) 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.561+0000 7fea18a00640 1 -- 172.21.15.43:0/1066303262 learned_addr learned my addr 172.21.15.43:0/1066303262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:32.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea19400640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea140b5b80 0x7fea140cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea19e00640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea1415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 -- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 msgr2=0x7fea1415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea1415ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 -- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea140b5b80 msgr2=0x7fea140cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea140b5b80 0x7fea140cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 -- 172.21.15.43:0/1066303262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea141590e0 con 0x7fea140d1620 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea18a00640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea00035040 tx=0x7fea000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.562+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00041070 con 0x7fea140d1620 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.563+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0003c460 con 0x7fea140d1620 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.563+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00037040 con 0x7fea140d1620 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.563+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea140cc400 con 0x7fea140d1620 2024-04-26T20:26:32.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.563+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea140cc9c0 con 0x7fea140d1620 2024-04-26T20:26:32.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.564+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea14069350 con 0x7fea140d1620 2024-04-26T20:26:32.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.565+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea0003a070 con 0x7fea140d1620 2024-04-26T20:26:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.566+0000 7fea11600640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 0x7fe9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.566+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fea000abfe0 con 0x7fea140d1620 2024-04-26T20:26:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.566+0000 7fea19400640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 0x7fe9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:32.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.567+0000 7fea19400640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 0x7fe9f406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea04005c00 tx=0x7fea04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.571+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea0007a100 con 0x7fea140d1620 2024-04-26T20:26:32.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.763+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea140008d0 con 0x7fe9f406c8f0 2024-04-26T20:26:32.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.787+0000 7fea11600640 1 -- 172.21.15.43:0/1066303262 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7fea140008d0 con 0x7fe9f406c8f0 2024-04-26T20:26:32.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 msgr2=0x7fe9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 0x7fe9f406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea04005c00 tx=0x7fea04009040 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 msgr2=0x7fea14158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fea00035040 tx=0x7fea000046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 shutdown_connections 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe9f406c8f0 0x7fe9f406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea140d2800 0x7fea1415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea140d1620 0x7fea14158ba0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 --2- 172.21.15.43:0/1066303262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea140b5b80 0x7fea140cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.794+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 >> 172.21.15.43:0/1066303262 conn(0x7fea140d4c30 msgr2=0x7fea140d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.795+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 shutdown_connections 2024-04-26T20:26:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:32.795+0000 7fea1a800640 1 -- 172.21.15.43:0/1066303262 wait complete. 2024-04-26T20:26:32.814 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-26T20:26:30.869891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227728,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746992,"statfs":{"total":773094113280,"available":772860919808,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005603"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347756+0000","last_change":"2024-04-26T20:25:56.984031+0000","last_active":"2024-04-26T20:26:03.347756+0000","last_peered":"2024-04-26T20:26:03.347756+0000","last_clean":"2024-04-26T20:26:03.347756+0000","last_became_active":"2024-04-26T20:25:56.983640+0000","last_became_peered":"2024-04-26T20:25:56.983640+0000","last_unstale":"2024-04-26T20:26:03.347756+0000","last_undegraded":"2024-04-26T20:26:03.347756+0000","last_fullsized":"2024-04-26T20:26:03.347756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:04.709694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094688+0000","last_change":"2024-04-26T20:25:57.999359+0000","last_active":"2024-04-26T20:26:03.094688+0000","last_peered":"2024-04-26T20:26:03.094688+0000","last_clean":"2024-04-26T20:26:03.094688+0000","last_became_active":"2024-04-26T20:25:57.999124+0000","last_became_peered":"2024-04-26T20:25:57.999124+0000","last_unstale":"2024-04-26T20:26:03.094688+0000","last_undegraded":"2024-04-26T20:26:03.094688+0000","last_fullsized":"2024-04-26T20:26:03.094688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092925+0000","last_change":"2024-04-26T20:26:02.417831+0000","last_active":"2024-04-26T20:26:03.092925+0000","last_peered":"2024-04-26T20:26:03.092925+0000","last_clean":"2024-04-26T20:26:03.092925+0000","last_became_active":"2024-04-26T20:26:02.417256+0000","last_became_peered":"2024-04-26T20:26:02.417256+0000","last_unstale":"2024-04-26T20:26:03.092925+0000","last_undegraded":"2024-04-26T20:26:03.092925+0000","last_fullsized":"2024-04-26T20:26:03.092925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094948+0000","last_change":"2024-04-26T20:25:56.981647+0000","last_active":"2024-04-26T20:26:03.094948+0000","last_peered":"2024-04-26T20:26:03.094948+0000","last_clean":"2024-04-26T20:26:03.094948+0000","last_became_active":"2024-04-26T20:25:56.980997+0000","last_became_peered":"2024-04-26T20:25:56.980997+0000","last_unstale":"2024-04-26T20:26:03.094948+0000","last_undegraded":"2024-04-26T20:26:03.094948+0000","last_fullsized":"2024-04-26T20:26:03.094948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347273+0000","last_change":"2024-04-26T20:25:58.006366+0000","last_active":"2024-04-26T20:26:03.347273+0000","last_peered":"2024-04-26T20:26:03.347273+0000","last_clean":"2024-04-26T20:26:03.347273+0000","last_became_active":"2024-04-26T20:25:58.005106+0000","last_became_peered":"2024-04-26T20:25:58.005106+0000","last_unstale":"2024-04-26T20:26:03.347273+0000","last_undegraded":"2024-04-26T20:26:03.347273+0000","last_fullsized":"2024-04-26T20:26:03.347273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050650+0000","last_change":"2024-04-26T20:26:02.077388+0000","last_active":"2024-04-26T20:26:03.050650+0000","last_peered":"2024-04-26T20:26:03.050650+0000","last_clean":"2024-04-26T20:26:03.050650+0000","last_became_active":"2024-04-26T20:26:02.076618+0000","last_became_peered":"2024-04-26T20:26:02.076618+0000","last_unstale":"2024-04-26T20:26:03.050650+0000","last_undegraded":"2024-04-26T20:26:03.050650+0000","last_fullsized":"2024-04-26T20:26:03.050650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:59.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348208+0000","last_change":"2024-04-26T20:25:56.993261+0000","last_active":"2024-04-26T20:26:03.348208+0000","last_peered":"2024-04-26T20:26:03.348208+0000","last_clean":"2024-04-26T20:26:03.348208+0000","last_became_active":"2024-04-26T20:25:56.992657+0000","last_became_peered":"2024-04-26T20:25:56.992657+0000","last_unstale":"2024-04-26T20:26:03.348208+0000","last_undegraded":"2024-04-26T20:26:03.348208+0000","last_fullsized":"2024-04-26T20:26:03.348208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346205+0000","last_change":"2024-04-26T20:25:58.008848+0000","last_active":"2024-04-26T20:26:03.346205+0000","last_peered":"2024-04-26T20:26:03.346205+0000","last_clean":"2024-04-26T20:26:03.346205+0000","last_became_active":"2024-04-26T20:25:58.008316+0000","last_became_peered":"2024-04-26T20:25:58.008316+0000","last_unstale":"2024-04-26T20:26:03.346205+0000","last_undegraded":"2024-04-26T20:26:03.346205+0000","last_fullsized":"2024-04-26T20:26:03.346205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051568+0000","last_change":"2024-04-26T20:26:02.080666+0000","last_active":"2024-04-26T20:26:03.051568+0000","last_peered":"2024-04-26T20:26:03.051568+0000","last_clean":"2024-04-26T20:26:03.051568+0000","last_became_active":"2024-04-26T20:26:02.080286+0000","last_became_peered":"2024-04-26T20:26:02.080286+0000","last_unstale":"2024-04-26T20:26:03.051568+0000","last_undegraded":"2024-04-26T20:26:03.051568+0000","last_fullsized":"2024-04-26T20:26:03.051568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347297+0000","last_change":"2024-04-26T20:25:56.993036+0000","last_active":"2024-04-26T20:26:03.347297+0000","last_peered":"2024-04-26T20:26:03.347297+0000","last_clean":"2024-04-26T20:26:03.347297+0000","last_became_active":"2024-04-26T20:25:56.992712+0000","last_became_peered":"2024-04-26T20:25:56.992712+0000","last_unstale":"2024-04-26T20:26:03.347297+0000","last_undegraded":"2024-04-26T20:26:03.347297+0000","last_fullsized":"2024-04-26T20:26:03.347297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:11.781113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347210+0000","last_change":"2024-04-26T20:25:58.005509+0000","last_active":"2024-04-26T20:26:03.347210+0000","last_peered":"2024-04-26T20:26:03.347210+0000","last_clean":"2024-04-26T20:26:03.347210+0000","last_became_active":"2024-04-26T20:25:58.005266+0000","last_became_peered":"2024-04-26T20:25:58.005266+0000","last_unstale":"2024-04-26T20:26:03.347210+0000","last_undegraded":"2024-04-26T20:26:03.347210+0000","last_fullsized":"2024-04-26T20:26:03.347210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050541+0000","last_change":"2024-04-26T20:26:02.076146+0000","last_active":"2024-04-26T20:26:03.050541+0000","last_peered":"2024-04-26T20:26:03.050541+0000","last_clean":"2024-04-26T20:26:03.050541+0000","last_became_active":"2024-04-26T20:26:02.075597+0000","last_became_peered":"2024-04-26T20:26:02.075597+0000","last_unstale":"2024-04-26T20:26:03.050541+0000","last_undegraded":"2024-04-26T20:26:03.050541+0000","last_fullsized":"2024-04-26T20:26:03.050541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:49.620083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349354+0000","last_change":"2024-04-26T20:25:57.995411+0000","last_active":"2024-04-26T20:26:03.349354+0000","last_peered":"2024-04-26T20:26:03.349354+0000","last_clean":"2024-04-26T20:26:03.349354+0000","last_became_active":"2024-04-26T20:25:57.993865+0000","last_became_peered":"2024-04-26T20:25:57.993865+0000","last_unstale":"2024-04-26T20:26:03.349354+0000","last_undegraded":"2024-04-26T20:26:03.349354+0000","last_fullsized":"2024-04-26T20:26:03.349354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347802+0000","last_change":"2024-04-26T20:25:56.986121+0000","last_active":"2024-04-26T20:26:03.347802+0000","last_peered":"2024-04-26T20:26:03.347802+0000","last_clean":"2024-04-26T20:26:03.347802+0000","last_became_active":"2024-04-26T20:25:56.985836+0000","last_became_peered":"2024-04-26T20:25:56.985836+0000","last_unstale":"2024-04-26T20:26:03.347802+0000","last_undegraded":"2024-04-26T20:26:03.347802+0000","last_fullsized":"2024-04-26T20:26:03.347802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:01.989205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049987+0000","last_change":"2024-04-26T20:26:02.079776+0000","last_active":"2024-04-26T20:26:03.049987+0000","last_peered":"2024-04-26T20:26:03.049987+0000","last_clean":"2024-04-26T20:26:03.049987+0000","last_became_active":"2024-04-26T20:26:02.079355+0000","last_became_peered":"2024-04-26T20:26:02.079355+0000","last_unstale":"2024-04-26T20:26:03.049987+0000","last_undegraded":"2024-04-26T20:26:03.049987+0000","last_fullsized":"2024-04-26T20:26:03.049987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:32.933389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093856+0000","last_change":"2024-04-26T20:25:57.990924+0000","last_active":"2024-04-26T20:26:03.093856+0000","last_peered":"2024-04-26T20:26:03.093856+0000","last_clean":"2024-04-26T20:26:03.093856+0000","last_became_active":"2024-04-26T20:25:57.990615+0000","last_became_peered":"2024-04-26T20:25:57.990615+0000","last_unstale":"2024-04-26T20:26:03.093856+0000","last_undegraded":"2024-04-26T20:26:03.093856+0000","last_fullsized":"2024-04-26T20:26:03.093856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345060+0000","last_change":"2024-04-26T20:25:56.985786+0000","last_active":"2024-04-26T20:26:03.345060+0000","last_peered":"2024-04-26T20:26:03.345060+0000","last_clean":"2024-04-26T20:26:03.345060+0000","last_became_active":"2024-04-26T20:25:56.981498+0000","last_became_peered":"2024-04-26T20:25:56.981498+0000","last_unstale":"2024-04-26T20:26:03.345060+0000","last_undegraded":"2024-04-26T20:26:03.345060+0000","last_fullsized":"2024-04-26T20:26:03.345060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049606+0000","last_change":"2024-04-26T20:26:02.075573+0000","last_active":"2024-04-26T20:26:03.049606+0000","last_peered":"2024-04-26T20:26:03.049606+0000","last_clean":"2024-04-26T20:26:03.049606+0000","last_became_active":"2024-04-26T20:26:02.075012+0000","last_became_peered":"2024-04-26T20:26:02.075012+0000","last_unstale":"2024-04-26T20:26:03.049606+0000","last_undegraded":"2024-04-26T20:26:03.049606+0000","last_fullsized":"2024-04-26T20:26:03.049606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:14.401666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093255+0000","last_change":"2024-04-26T20:25:58.011840+0000","last_active":"2024-04-26T20:26:03.093255+0000","last_peered":"2024-04-26T20:26:03.093255+0000","last_clean":"2024-04-26T20:26:03.093255+0000","last_became_active":"2024-04-26T20:25:58.011346+0000","last_became_peered":"2024-04-26T20:25:58.011346+0000","last_unstale":"2024-04-26T20:26:03.093255+0000","last_undegraded":"2024-04-26T20:26:03.093255+0000","last_fullsized":"2024-04-26T20:26:03.093255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344179+0000","last_change":"2024-04-26T20:25:56.986011+0000","last_active":"2024-04-26T20:26:03.344179+0000","last_peered":"2024-04-26T20:26:03.344179+0000","last_clean":"2024-04-26T20:26:03.344179+0000","last_became_active":"2024-04-26T20:25:56.985728+0000","last_became_peered":"2024-04-26T20:25:56.985728+0000","last_unstale":"2024-04-26T20:26:03.344179+0000","last_undegraded":"2024-04-26T20:26:03.344179+0000","last_fullsized":"2024-04-26T20:26:03.344179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347438+0000","last_change":"2024-04-26T20:26:02.401041+0000","last_active":"2024-04-26T20:26:03.347438+0000","last_peered":"2024-04-26T20:26:03.347438+0000","last_clean":"2024-04-26T20:26:03.347438+0000","last_became_active":"2024-04-26T20:26:02.400108+0000","last_became_peered":"2024-04-26T20:26:02.400108+0000","last_unstale":"2024-04-26T20:26:03.347438+0000","last_undegraded":"2024-04-26T20:26:03.347438+0000","last_fullsized":"2024-04-26T20:26:03.347438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049644+0000","last_change":"2024-04-26T20:25:57.985787+0000","last_active":"2024-04-26T20:26:03.049644+0000","last_peered":"2024-04-26T20:26:03.049644+0000","last_clean":"2024-04-26T20:26:03.049644+0000","last_became_active":"2024-04-26T20:25:57.985432+0000","last_became_peered":"2024-04-26T20:25:57.985432+0000","last_unstale":"2024-04-26T20:26:03.049644+0000","last_undegraded":"2024-04-26T20:26:03.049644+0000","last_fullsized":"2024-04-26T20:26:03.049644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:37.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345394+0000","last_change":"2024-04-26T20:25:56.983944+0000","last_active":"2024-04-26T20:26:03.345394+0000","last_peered":"2024-04-26T20:26:03.345394+0000","last_clean":"2024-04-26T20:26:03.345394+0000","last_became_active":"2024-04-26T20:25:56.983642+0000","last_became_peered":"2024-04-26T20:25:56.983642+0000","last_unstale":"2024-04-26T20:26:03.345394+0000","last_undegraded":"2024-04-26T20:26:03.345394+0000","last_fullsized":"2024-04-26T20:26:03.345394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:12.976926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050080+0000","last_change":"2024-04-26T20:26:02.082133+0000","last_active":"2024-04-26T20:26:03.050080+0000","last_peered":"2024-04-26T20:26:03.050080+0000","last_clean":"2024-04-26T20:26:03.050080+0000","last_became_active":"2024-04-26T20:26:02.081547+0000","last_became_peered":"2024-04-26T20:26:02.081547+0000","last_unstale":"2024-04-26T20:26:03.050080+0000","last_undegraded":"2024-04-26T20:26:03.050080+0000","last_fullsized":"2024-04-26T20:26:03.050080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:39.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347003+0000","last_change":"2024-04-26T20:25:57.989909+0000","last_active":"2024-04-26T20:26:03.347003+0000","last_peered":"2024-04-26T20:26:03.347003+0000","last_clean":"2024-04-26T20:26:03.347003+0000","last_became_active":"2024-04-26T20:25:57.989626+0000","last_became_peered":"2024-04-26T20:25:57.989626+0000","last_unstale":"2024-04-26T20:26:03.347003+0000","last_undegraded":"2024-04-26T20:26:03.347003+0000","last_fullsized":"2024-04-26T20:26:03.347003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351956+0000","last_change":"2024-04-26T20:25:56.985557+0000","last_active":"2024-04-26T20:26:03.351956+0000","last_peered":"2024-04-26T20:26:03.351956+0000","last_clean":"2024-04-26T20:26:03.351956+0000","last_became_active":"2024-04-26T20:25:56.985217+0000","last_became_peered":"2024-04-26T20:25:56.985217+0000","last_unstale":"2024-04-26T20:26:03.351956+0000","last_undegraded":"2024-04-26T20:26:03.351956+0000","last_fullsized":"2024-04-26T20:26:03.351956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050981+0000","last_change":"2024-04-26T20:26:02.405217+0000","last_active":"2024-04-26T20:26:03.050981+0000","last_peered":"2024-04-26T20:26:03.050981+0000","last_clean":"2024-04-26T20:26:03.050981+0000","last_became_active":"2024-04-26T20:26:02.404124+0000","last_became_peered":"2024-04-26T20:26:02.404124+0000","last_unstale":"2024-04-26T20:26:03.050981+0000","last_undegraded":"2024-04-26T20:26:03.050981+0000","last_fullsized":"2024-04-26T20:26:03.050981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:29.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095109+0000","last_change":"2024-04-26T20:25:57.999075+0000","last_active":"2024-04-26T20:26:03.095109+0000","last_peered":"2024-04-26T20:26:03.095109+0000","last_clean":"2024-04-26T20:26:03.095109+0000","last_became_active":"2024-04-26T20:25:57.998666+0000","last_became_peered":"2024-04-26T20:25:57.998666+0000","last_unstale":"2024-04-26T20:26:03.095109+0000","last_undegraded":"2024-04-26T20:26:03.095109+0000","last_fullsized":"2024-04-26T20:26:03.095109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:38:25.779495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346736+0000","last_change":"2024-04-26T20:25:56.977150+0000","last_active":"2024-04-26T20:26:03.346736+0000","last_peered":"2024-04-26T20:26:03.346736+0000","last_clean":"2024-04-26T20:26:03.346736+0000","last_became_active":"2024-04-26T20:25:56.976895+0000","last_became_peered":"2024-04-26T20:25:56.976895+0000","last_unstale":"2024-04-26T20:26:03.346736+0000","last_undegraded":"2024-04-26T20:26:03.346736+0000","last_fullsized":"2024-04-26T20:26:03.346736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093148+0000","last_change":"2024-04-26T20:26:02.400496+0000","last_active":"2024-04-26T20:26:03.093148+0000","last_peered":"2024-04-26T20:26:03.093148+0000","last_clean":"2024-04-26T20:26:03.093148+0000","last_became_active":"2024-04-26T20:26:02.400002+0000","last_became_peered":"2024-04-26T20:26:02.400002+0000","last_unstale":"2024-04-26T20:26:03.093148+0000","last_undegraded":"2024-04-26T20:26:03.093148+0000","last_fullsized":"2024-04-26T20:26:03.093148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345843+0000","last_change":"2024-04-26T20:25:58.009127+0000","last_active":"2024-04-26T20:26:03.345843+0000","last_peered":"2024-04-26T20:26:03.345843+0000","last_clean":"2024-04-26T20:26:03.345843+0000","last_became_active":"2024-04-26T20:25:58.006542+0000","last_became_peered":"2024-04-26T20:25:58.006542+0000","last_unstale":"2024-04-26T20:26:03.345843+0000","last_undegraded":"2024-04-26T20:26:03.345843+0000","last_fullsized":"2024-04-26T20:26:03.345843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348511+0000","last_change":"2024-04-26T20:25:56.992958+0000","last_active":"2024-04-26T20:26:03.348511+0000","last_peered":"2024-04-26T20:26:03.348511+0000","last_clean":"2024-04-26T20:26:03.348511+0000","last_became_active":"2024-04-26T20:25:56.992326+0000","last_became_peered":"2024-04-26T20:25:56.992326+0000","last_unstale":"2024-04-26T20:26:03.348511+0000","last_undegraded":"2024-04-26T20:26:03.348511+0000","last_fullsized":"2024-04-26T20:26:03.348511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346294+0000","last_change":"2024-04-26T20:26:02.406897+0000","last_active":"2024-04-26T20:26:03.346294+0000","last_peered":"2024-04-26T20:26:03.346294+0000","last_clean":"2024-04-26T20:26:03.346294+0000","last_became_active":"2024-04-26T20:26:02.402553+0000","last_became_peered":"2024-04-26T20:26:02.402553+0000","last_unstale":"2024-04-26T20:26:03.346294+0000","last_undegraded":"2024-04-26T20:26:03.346294+0000","last_fullsized":"2024-04-26T20:26:03.346294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347458+0000","last_change":"2024-04-26T20:25:58.003889+0000","last_active":"2024-04-26T20:26:03.347458+0000","last_peered":"2024-04-26T20:26:03.347458+0000","last_clean":"2024-04-26T20:26:03.347458+0000","last_became_active":"2024-04-26T20:25:58.003578+0000","last_became_peered":"2024-04-26T20:25:58.003578+0000","last_unstale":"2024-04-26T20:26:03.347458+0000","last_undegraded":"2024-04-26T20:26:03.347458+0000","last_fullsized":"2024-04-26T20:26:03.347458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346054+0000","last_change":"2024-04-26T20:25:56.975939+0000","last_active":"2024-04-26T20:26:03.346054+0000","last_peered":"2024-04-26T20:26:03.346054+0000","last_clean":"2024-04-26T20:26:03.346054+0000","last_became_active":"2024-04-26T20:25:56.975669+0000","last_became_peered":"2024-04-26T20:25:56.975669+0000","last_unstale":"2024-04-26T20:26:03.346054+0000","last_undegraded":"2024-04-26T20:26:03.346054+0000","last_fullsized":"2024-04-26T20:26:03.346054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:47.272220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344705+0000","last_change":"2024-04-26T20:26:02.079073+0000","last_active":"2024-04-26T20:26:03.344705+0000","last_peered":"2024-04-26T20:26:03.344705+0000","last_clean":"2024-04-26T20:26:03.344705+0000","last_became_active":"2024-04-26T20:26:02.074779+0000","last_became_peered":"2024-04-26T20:26:02.074779+0000","last_unstale":"2024-04-26T20:26:03.344705+0000","last_undegraded":"2024-04-26T20:26:03.344705+0000","last_fullsized":"2024-04-26T20:26:03.344705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051836+0000","last_change":"2024-04-26T20:25:57.988308+0000","last_active":"2024-04-26T20:26:03.051836+0000","last_peered":"2024-04-26T20:26:03.051836+0000","last_clean":"2024-04-26T20:26:03.051836+0000","last_became_active":"2024-04-26T20:25:57.984254+0000","last_became_peered":"2024-04-26T20:25:57.984254+0000","last_unstale":"2024-04-26T20:26:03.051836+0000","last_undegraded":"2024-04-26T20:26:03.051836+0000","last_fullsized":"2024-04-26T20:26:03.051836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:26.721241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348724+0000","last_change":"2024-04-26T20:25:56.980841+0000","last_active":"2024-04-26T20:26:03.348724+0000","last_peered":"2024-04-26T20:26:03.348724+0000","last_clean":"2024-04-26T20:26:03.348724+0000","last_became_active":"2024-04-26T20:25:56.980215+0000","last_became_peered":"2024-04-26T20:25:56.980215+0000","last_unstale":"2024-04-26T20:26:03.348724+0000","last_undegraded":"2024-04-26T20:26:03.348724+0000","last_fullsized":"2024-04-26T20:26:03.348724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345720+0000","last_change":"2024-04-26T20:26:02.078478+0000","last_active":"2024-04-26T20:26:03.345720+0000","last_peered":"2024-04-26T20:26:03.345720+0000","last_clean":"2024-04-26T20:26:03.345720+0000","last_became_active":"2024-04-26T20:26:02.078121+0000","last_became_peered":"2024-04-26T20:26:02.078121+0000","last_unstale":"2024-04-26T20:26:03.345720+0000","last_undegraded":"2024-04-26T20:26:03.345720+0000","last_fullsized":"2024-04-26T20:26:03.345720+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343920+0000","last_change":"2024-04-26T20:25:58.001121+0000","last_active":"2024-04-26T20:26:03.343920+0000","last_peered":"2024-04-26T20:26:03.343920+0000","last_clean":"2024-04-26T20:26:03.343920+0000","last_became_active":"2024-04-26T20:25:58.000412+0000","last_became_peered":"2024-04-26T20:25:58.000412+0000","last_unstale":"2024-04-26T20:26:03.343920+0000","last_undegraded":"2024-04-26T20:26:03.343920+0000","last_fullsized":"2024-04-26T20:26:03.343920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048086+0000","last_change":"2024-04-26T20:25:56.972300+0000","last_active":"2024-04-26T20:26:03.048086+0000","last_peered":"2024-04-26T20:26:03.048086+0000","last_clean":"2024-04-26T20:26:03.048086+0000","last_became_active":"2024-04-26T20:25:56.972020+0000","last_became_peered":"2024-04-26T20:25:56.972020+0000","last_unstale":"2024-04-26T20:26:03.048086+0000","last_undegraded":"2024-04-26T20:26:03.048086+0000","last_fullsized":"2024-04-26T20:26:03.048086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347521+0000","last_change":"2024-04-26T20:26:02.077923+0000","last_active":"2024-04-26T20:26:03.347521+0000","last_peered":"2024-04-26T20:26:03.347521+0000","last_clean":"2024-04-26T20:26:03.347521+0000","last_became_active":"2024-04-26T20:26:02.077483+0000","last_became_peered":"2024-04-26T20:26:02.077483+0000","last_unstale":"2024-04-26T20:26:03.347521+0000","last_undegraded":"2024-04-26T20:26:03.347521+0000","last_fullsized":"2024-04-26T20:26:03.347521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344678+0000","last_change":"2024-04-26T20:25:57.984633+0000","last_active":"2024-04-26T20:26:03.344678+0000","last_peered":"2024-04-26T20:26:03.344678+0000","last_clean":"2024-04-26T20:26:03.344678+0000","last_became_active":"2024-04-26T20:25:57.984262+0000","last_became_peered":"2024-04-26T20:25:57.984262+0000","last_unstale":"2024-04-26T20:26:03.344678+0000","last_undegraded":"2024-04-26T20:26:03.344678+0000","last_fullsized":"2024-04-26T20:26:03.344678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:03.002879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345671+0000","last_change":"2024-04-26T20:25:56.989467+0000","last_active":"2024-04-26T20:26:03.345671+0000","last_peered":"2024-04-26T20:26:03.345671+0000","last_clean":"2024-04-26T20:26:03.345671+0000","last_became_active":"2024-04-26T20:25:56.989183+0000","last_became_peered":"2024-04-26T20:25:56.989183+0000","last_unstale":"2024-04-26T20:26:03.345671+0000","last_undegraded":"2024-04-26T20:26:03.345671+0000","last_fullsized":"2024-04-26T20:26:03.345671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:11.650840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346929+0000","last_change":"2024-04-26T20:26:02.414455+0000","last_active":"2024-04-26T20:26:03.346929+0000","last_peered":"2024-04-26T20:26:03.346929+0000","last_clean":"2024-04-26T20:26:03.346929+0000","last_became_active":"2024-04-26T20:26:02.414064+0000","last_became_peered":"2024-04-26T20:26:02.414064+0000","last_unstale":"2024-04-26T20:26:03.346929+0000","last_undegraded":"2024-04-26T20:26:03.346929+0000","last_fullsized":"2024-04-26T20:26:03.346929+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:26.167323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093496+0000","last_change":"2024-04-26T20:25:58.008847+0000","last_active":"2024-04-26T20:26:03.093496+0000","last_peered":"2024-04-26T20:26:03.093496+0000","last_clean":"2024-04-26T20:26:03.093496+0000","last_became_active":"2024-04-26T20:25:58.008549+0000","last_became_peered":"2024-04-26T20:25:58.008549+0000","last_unstale":"2024-04-26T20:26:03.093496+0000","last_undegraded":"2024-04-26T20:26:03.093496+0000","last_fullsized":"2024-04-26T20:26:03.093496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345379+0000","last_change":"2024-04-26T20:25:56.981771+0000","last_active":"2024-04-26T20:26:03.345379+0000","last_peered":"2024-04-26T20:26:03.345379+0000","last_clean":"2024-04-26T20:26:03.345379+0000","last_became_active":"2024-04-26T20:25:56.981406+0000","last_became_peered":"2024-04-26T20:25:56.981406+0000","last_unstale":"2024-04-26T20:26:03.345379+0000","last_undegraded":"2024-04-26T20:26:03.345379+0000","last_fullsized":"2024-04-26T20:26:03.345379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345981+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:26:03.345981+0000","last_peered":"2024-04-26T20:26:03.345981+0000","last_clean":"2024-04-26T20:26:03.345981+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:26:03.345981+0000","last_undegraded":"2024-04-26T20:26:03.345981+0000","last_fullsized":"2024-04-26T20:26:03.345981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351315+0000","last_change":"2024-04-26T20:26:02.407562+0000","last_active":"2024-04-26T20:26:03.351315+0000","last_peered":"2024-04-26T20:26:03.351315+0000","last_clean":"2024-04-26T20:26:03.351315+0000","last_became_active":"2024-04-26T20:26:02.406288+0000","last_became_peered":"2024-04-26T20:26:02.406288+0000","last_unstale":"2024-04-26T20:26:03.351315+0000","last_undegraded":"2024-04-26T20:26:03.351315+0000","last_fullsized":"2024-04-26T20:26:03.351315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347732+0000","last_change":"2024-04-26T20:25:58.014416+0000","last_active":"2024-04-26T20:26:03.347732+0000","last_peered":"2024-04-26T20:26:03.347732+0000","last_clean":"2024-04-26T20:26:03.347732+0000","last_became_active":"2024-04-26T20:25:58.014128+0000","last_became_peered":"2024-04-26T20:25:58.014128+0000","last_unstale":"2024-04-26T20:26:03.347732+0000","last_undegraded":"2024-04-26T20:26:03.347732+0000","last_fullsized":"2024-04-26T20:26:03.347732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:22.089301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048942+0000","last_change":"2024-04-26T20:25:56.974272+0000","last_active":"2024-04-26T20:26:03.048942+0000","last_peered":"2024-04-26T20:26:03.048942+0000","last_clean":"2024-04-26T20:26:03.048942+0000","last_became_active":"2024-04-26T20:25:56.972999+0000","last_became_peered":"2024-04-26T20:25:56.972999+0000","last_unstale":"2024-04-26T20:26:03.048942+0000","last_undegraded":"2024-04-26T20:26:03.048942+0000","last_fullsized":"2024-04-26T20:26:03.048942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349430+0000","last_change":"2024-04-26T20:26:02.402839+0000","last_active":"2024-04-26T20:26:03.349430+0000","last_peered":"2024-04-26T20:26:03.349430+0000","last_clean":"2024-04-26T20:26:03.349430+0000","last_became_active":"2024-04-26T20:26:02.401630+0000","last_became_peered":"2024-04-26T20:26:02.401630+0000","last_unstale":"2024-04-26T20:26:03.349430+0000","last_undegraded":"2024-04-26T20:26:03.349430+0000","last_fullsized":"2024-04-26T20:26:03.349430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:55.514747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346830+0000","last_change":"2024-04-26T20:25:58.009012+0000","last_active":"2024-04-26T20:26:03.346830+0000","last_peered":"2024-04-26T20:26:03.346830+0000","last_clean":"2024-04-26T20:26:03.346830+0000","last_became_active":"2024-04-26T20:25:58.008594+0000","last_became_peered":"2024-04-26T20:25:58.008594+0000","last_unstale":"2024-04-26T20:26:03.346830+0000","last_undegraded":"2024-04-26T20:26:03.346830+0000","last_fullsized":"2024-04-26T20:26:03.346830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344764+0000","last_change":"2024-04-26T20:25:56.972043+0000","last_active":"2024-04-26T20:26:03.344764+0000","last_peered":"2024-04-26T20:26:03.344764+0000","last_clean":"2024-04-26T20:26:03.344764+0000","last_became_active":"2024-04-26T20:25:56.971671+0000","last_became_peered":"2024-04-26T20:25:56.971671+0000","last_unstale":"2024-04-26T20:26:03.344764+0000","last_undegraded":"2024-04-26T20:26:03.344764+0000","last_fullsized":"2024-04-26T20:26:03.344764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346513+0000","last_change":"2024-04-26T20:26:02.073728+0000","last_active":"2024-04-26T20:26:03.346513+0000","last_peered":"2024-04-26T20:26:03.346513+0000","last_clean":"2024-04-26T20:26:03.346513+0000","last_became_active":"2024-04-26T20:26:02.073238+0000","last_became_peered":"2024-04-26T20:26:02.073238+0000","last_unstale":"2024-04-26T20:26:03.346513+0000","last_undegraded":"2024-04-26T20:26:03.346513+0000","last_fullsized":"2024-04-26T20:26:03.346513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:32.243742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350204+0000","last_change":"2024-04-26T20:25:58.011539+0000","last_active":"2024-04-26T20:26:03.350204+0000","last_peered":"2024-04-26T20:26:03.350204+0000","last_clean":"2024-04-26T20:26:03.350204+0000","last_became_active":"2024-04-26T20:25:58.011256+0000","last_became_peered":"2024-04-26T20:25:58.011256+0000","last_unstale":"2024-04-26T20:26:03.350204+0000","last_undegraded":"2024-04-26T20:26:03.350204+0000","last_fullsized":"2024-04-26T20:26:03.350204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:02.292377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048755+0000","last_change":"2024-04-26T20:25:56.972332+0000","last_active":"2024-04-26T20:26:03.048755+0000","last_peered":"2024-04-26T20:26:03.048755+0000","last_clean":"2024-04-26T20:26:03.048755+0000","last_became_active":"2024-04-26T20:25:56.972075+0000","last_became_peered":"2024-04-26T20:25:56.972075+0000","last_unstale":"2024-04-26T20:26:03.048755+0000","last_undegraded":"2024-04-26T20:26:03.048755+0000","last_fullsized":"2024-04-26T20:26:03.048755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346282+0000","last_change":"2024-04-26T20:26:02.414970+0000","last_active":"2024-04-26T20:26:03.346282+0000","last_peered":"2024-04-26T20:26:03.346282+0000","last_clean":"2024-04-26T20:26:03.346282+0000","last_became_active":"2024-04-26T20:26:02.414093+0000","last_became_peered":"2024-04-26T20:26:02.414093+0000","last_unstale":"2024-04-26T20:26:03.346282+0000","last_undegraded":"2024-04-26T20:26:03.346282+0000","last_fullsized":"2024-04-26T20:26:03.346282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:14.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348952+0000","last_change":"2024-04-26T20:25:58.014604+0000","last_active":"2024-04-26T20:26:03.348952+0000","last_peered":"2024-04-26T20:26:03.348952+0000","last_clean":"2024-04-26T20:26:03.348952+0000","last_became_active":"2024-04-26T20:25:58.014239+0000","last_became_peered":"2024-04-26T20:25:58.014239+0000","last_unstale":"2024-04-26T20:26:03.348952+0000","last_undegraded":"2024-04-26T20:26:03.348952+0000","last_fullsized":"2024-04-26T20:26:03.348952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:24.555699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050237+0000","last_change":"2024-04-26T20:25:56.970496+0000","last_active":"2024-04-26T20:26:03.050237+0000","last_peered":"2024-04-26T20:26:03.050237+0000","last_clean":"2024-04-26T20:26:03.050237+0000","last_became_active":"2024-04-26T20:25:56.970174+0000","last_became_peered":"2024-04-26T20:25:56.970174+0000","last_unstale":"2024-04-26T20:26:03.050237+0000","last_undegraded":"2024-04-26T20:26:03.050237+0000","last_fullsized":"2024-04-26T20:26:03.050237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:25.599135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.049091+0000","last_change":"2024-04-26T20:26:02.082060+0000","last_active":"2024-04-26T20:26:03.049091+0000","last_peered":"2024-04-26T20:26:03.049091+0000","last_clean":"2024-04-26T20:26:03.049091+0000","last_became_active":"2024-04-26T20:26:02.081533+0000","last_became_peered":"2024-04-26T20:26:02.081533+0000","last_unstale":"2024-04-26T20:26:03.049091+0000","last_undegraded":"2024-04-26T20:26:03.049091+0000","last_fullsized":"2024-04-26T20:26:03.049091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:36:02.899043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346324+0000","last_change":"2024-04-26T20:25:58.008801+0000","last_active":"2024-04-26T20:26:03.346324+0000","last_peered":"2024-04-26T20:26:03.346324+0000","last_clean":"2024-04-26T20:26:03.346324+0000","last_became_active":"2024-04-26T20:25:58.008479+0000","last_became_peered":"2024-04-26T20:25:58.008479+0000","last_unstale":"2024-04-26T20:26:03.346324+0000","last_undegraded":"2024-04-26T20:26:03.346324+0000","last_fullsized":"2024-04-26T20:26:03.346324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047755+0000","last_change":"2024-04-26T20:25:56.968431+0000","last_active":"2024-04-26T20:26:03.047755+0000","last_peered":"2024-04-26T20:26:03.047755+0000","last_clean":"2024-04-26T20:26:03.047755+0000","last_became_active":"2024-04-26T20:25:56.967495+0000","last_became_peered":"2024-04-26T20:25:56.967495+0000","last_unstale":"2024-04-26T20:26:03.047755+0000","last_undegraded":"2024-04-26T20:26:03.047755+0000","last_fullsized":"2024-04-26T20:26:03.047755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346734+0000","last_change":"2024-04-26T20:26:02.078801+0000","last_active":"2024-04-26T20:26:03.346734+0000","last_peered":"2024-04-26T20:26:03.346734+0000","last_clean":"2024-04-26T20:26:03.346734+0000","last_became_active":"2024-04-26T20:26:02.078343+0000","last_became_peered":"2024-04-26T20:26:02.078343+0000","last_unstale":"2024-04-26T20:26:03.346734+0000","last_undegraded":"2024-04-26T20:26:03.346734+0000","last_fullsized":"2024-04-26T20:26:03.346734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:29.356103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350880+0000","last_change":"2024-04-26T20:25:57.996180+0000","last_active":"2024-04-26T20:26:03.350880+0000","last_peered":"2024-04-26T20:26:03.350880+0000","last_clean":"2024-04-26T20:26:03.350880+0000","last_became_active":"2024-04-26T20:25:57.990792+0000","last_became_peered":"2024-04-26T20:25:57.990792+0000","last_unstale":"2024-04-26T20:26:03.350880+0000","last_undegraded":"2024-04-26T20:26:03.350880+0000","last_fullsized":"2024-04-26T20:26:03.350880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349887+0000","last_change":"2024-04-26T20:25:56.974115+0000","last_active":"2024-04-26T20:26:03.349887+0000","last_peered":"2024-04-26T20:26:03.349887+0000","last_clean":"2024-04-26T20:26:03.349887+0000","last_became_active":"2024-04-26T20:25:56.973801+0000","last_became_peered":"2024-04-26T20:25:56.973801+0000","last_unstale":"2024-04-26T20:26:03.349887+0000","last_undegraded":"2024-04-26T20:26:03.349887+0000","last_fullsized":"2024-04-26T20:26:03.349887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:56:25.707567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.095178+0000","last_change":"2024-04-26T20:26:02.413590+0000","last_active":"2024-04-26T20:26:03.095178+0000","last_peered":"2024-04-26T20:26:03.095178+0000","last_clean":"2024-04-26T20:26:03.095178+0000","last_became_active":"2024-04-26T20:26:02.413104+0000","last_became_peered":"2024-04-26T20:26:02.413104+0000","last_unstale":"2024-04-26T20:26:03.095178+0000","last_undegraded":"2024-04-26T20:26:03.095178+0000","last_fullsized":"2024-04-26T20:26:03.095178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:35.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347726+0000","last_change":"2024-04-26T20:25:58.014095+0000","last_active":"2024-04-26T20:26:03.347726+0000","last_peered":"2024-04-26T20:26:03.347726+0000","last_clean":"2024-04-26T20:26:03.347726+0000","last_became_active":"2024-04-26T20:25:58.012813+0000","last_became_peered":"2024-04-26T20:25:58.012813+0000","last_unstale":"2024-04-26T20:26:03.347726+0000","last_undegraded":"2024-04-26T20:26:03.347726+0000","last_fullsized":"2024-04-26T20:26:03.347726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348384+0000","last_change":"2024-04-26T20:25:56.987103+0000","last_active":"2024-04-26T20:26:03.348384+0000","last_peered":"2024-04-26T20:26:03.348384+0000","last_clean":"2024-04-26T20:26:03.348384+0000","last_became_active":"2024-04-26T20:25:56.986766+0000","last_became_peered":"2024-04-26T20:25:56.986766+0000","last_unstale":"2024-04-26T20:26:03.348384+0000","last_undegraded":"2024-04-26T20:26:03.348384+0000","last_fullsized":"2024-04-26T20:26:03.348384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094102+0000","last_change":"2024-04-26T20:26:02.400999+0000","last_active":"2024-04-26T20:26:03.094102+0000","last_peered":"2024-04-26T20:26:03.094102+0000","last_clean":"2024-04-26T20:26:03.094102+0000","last_became_active":"2024-04-26T20:26:02.399994+0000","last_became_peered":"2024-04-26T20:26:02.399994+0000","last_unstale":"2024-04-26T20:26:03.094102+0000","last_undegraded":"2024-04-26T20:26:03.094102+0000","last_fullsized":"2024-04-26T20:26:03.094102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:05.311868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345529+0000","last_change":"2024-04-26T20:25:57.996909+0000","last_active":"2024-04-26T20:26:03.345529+0000","last_peered":"2024-04-26T20:26:03.345529+0000","last_clean":"2024-04-26T20:26:03.345529+0000","last_became_active":"2024-04-26T20:25:57.996635+0000","last_became_peered":"2024-04-26T20:25:57.996635+0000","last_unstale":"2024-04-26T20:26:03.345529+0000","last_undegraded":"2024-04-26T20:26:03.345529+0000","last_fullsized":"2024-04-26T20:26:03.345529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346908+0000","last_change":"2024-04-26T20:25:56.974019+0000","last_active":"2024-04-26T20:26:03.346908+0000","last_peered":"2024-04-26T20:26:03.346908+0000","last_clean":"2024-04-26T20:26:03.346908+0000","last_became_active":"2024-04-26T20:25:56.971170+0000","last_became_peered":"2024-04-26T20:25:56.971170+0000","last_unstale":"2024-04-26T20:26:03.346908+0000","last_undegraded":"2024-04-26T20:26:03.346908+0000","last_fullsized":"2024-04-26T20:26:03.346908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347253+0000","last_change":"2024-04-26T20:26:02.402012+0000","last_active":"2024-04-26T20:26:03.347253+0000","last_peered":"2024-04-26T20:26:03.347253+0000","last_clean":"2024-04-26T20:26:03.347253+0000","last_became_active":"2024-04-26T20:26:02.400605+0000","last_became_peered":"2024-04-26T20:26:02.400605+0000","last_unstale":"2024-04-26T20:26:03.347253+0000","last_undegraded":"2024-04-26T20:26:03.347253+0000","last_fullsized":"2024-04-26T20:26:03.347253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:32.688333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347168+0000","last_change":"2024-04-26T20:25:58.001764+0000","last_active":"2024-04-26T20:26:03.347168+0000","last_peered":"2024-04-26T20:26:03.347168+0000","last_clean":"2024-04-26T20:26:03.347168+0000","last_became_active":"2024-04-26T20:25:58.001502+0000","last_became_peered":"2024-04-26T20:25:58.001502+0000","last_unstale":"2024-04-26T20:26:03.347168+0000","last_undegraded":"2024-04-26T20:26:03.347168+0000","last_fullsized":"2024-04-26T20:26:03.347168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:25.278081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:16.030394+0000","last_change":"2024-04-26T20:25:56.984259+0000","last_active":"2024-04-26T20:26:16.030394+0000","last_peered":"2024-04-26T20:26:16.030394+0000","last_clean":"2024-04-26T20:26:16.030394+0000","last_became_active":"2024-04-26T20:25:56.983974+0000","last_became_peered":"2024-04-26T20:25:56.983974+0000","last_unstale":"2024-04-26T20:26:16.030394+0000","last_undegraded":"2024-04-26T20:26:16.030394+0000","last_fullsized":"2024-04-26T20:26:16.030394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:44.211963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345142+0000","last_change":"2024-04-26T20:26:02.080802+0000","last_active":"2024-04-26T20:26:03.345142+0000","last_peered":"2024-04-26T20:26:03.345142+0000","last_clean":"2024-04-26T20:26:03.345142+0000","last_became_active":"2024-04-26T20:26:02.079790+0000","last_became_peered":"2024-04-26T20:26:02.079790+0000","last_unstale":"2024-04-26T20:26:03.345142+0000","last_undegraded":"2024-04-26T20:26:03.345142+0000","last_fullsized":"2024-04-26T20:26:03.345142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050516+0000","last_change":"2024-04-26T20:25:57.999695+0000","last_active":"2024-04-26T20:26:03.050516+0000","last_peered":"2024-04-26T20:26:03.050516+0000","last_clean":"2024-04-26T20:26:03.050516+0000","last_became_active":"2024-04-26T20:25:57.999476+0000","last_became_peered":"2024-04-26T20:25:57.999476+0000","last_unstale":"2024-04-26T20:26:03.050516+0000","last_undegraded":"2024-04-26T20:26:03.050516+0000","last_fullsized":"2024-04-26T20:26:03.050516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344567+0000","last_change":"2024-04-26T20:25:56.984463+0000","last_active":"2024-04-26T20:26:03.344567+0000","last_peered":"2024-04-26T20:26:03.344567+0000","last_clean":"2024-04-26T20:26:03.344567+0000","last_became_active":"2024-04-26T20:25:56.984176+0000","last_became_peered":"2024-04-26T20:25:56.984176+0000","last_unstale":"2024-04-26T20:26:03.344567+0000","last_undegraded":"2024-04-26T20:26:03.344567+0000","last_fullsized":"2024-04-26T20:26:03.344567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:33.657620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.051617+0000","last_change":"2024-04-26T20:26:02.079338+0000","last_active":"2024-04-26T20:26:03.051617+0000","last_peered":"2024-04-26T20:26:03.051617+0000","last_clean":"2024-04-26T20:26:03.051617+0000","last_became_active":"2024-04-26T20:26:02.078830+0000","last_became_peered":"2024-04-26T20:26:02.078830+0000","last_unstale":"2024-04-26T20:26:03.051617+0000","last_undegraded":"2024-04-26T20:26:03.051617+0000","last_fullsized":"2024-04-26T20:26:03.051617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:20.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347553+0000","last_change":"2024-04-26T20:25:58.008157+0000","last_active":"2024-04-26T20:26:03.347553+0000","last_peered":"2024-04-26T20:26:03.347553+0000","last_clean":"2024-04-26T20:26:03.347553+0000","last_became_active":"2024-04-26T20:25:58.005542+0000","last_became_peered":"2024-04-26T20:25:58.005542+0000","last_unstale":"2024-04-26T20:26:03.347553+0000","last_undegraded":"2024-04-26T20:26:03.347553+0000","last_fullsized":"2024-04-26T20:26:03.347553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348327+0000","last_change":"2024-04-26T20:25:56.987547+0000","last_active":"2024-04-26T20:26:03.348327+0000","last_peered":"2024-04-26T20:26:03.348327+0000","last_clean":"2024-04-26T20:26:03.348327+0000","last_became_active":"2024-04-26T20:25:56.985375+0000","last_became_peered":"2024-04-26T20:25:56.985375+0000","last_unstale":"2024-04-26T20:26:03.348327+0000","last_undegraded":"2024-04-26T20:26:03.348327+0000","last_fullsized":"2024-04-26T20:26:03.348327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:28.544478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093045+0000","last_change":"2024-04-26T20:26:02.073521+0000","last_active":"2024-04-26T20:26:03.093045+0000","last_peered":"2024-04-26T20:26:03.093045+0000","last_clean":"2024-04-26T20:26:03.093045+0000","last_became_active":"2024-04-26T20:26:02.073134+0000","last_became_peered":"2024-04-26T20:26:02.073134+0000","last_unstale":"2024-04-26T20:26:03.093045+0000","last_undegraded":"2024-04-26T20:26:03.093045+0000","last_fullsized":"2024-04-26T20:26:03.093045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:06.370398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348491+0000","last_change":"2024-04-26T20:25:58.011178+0000","last_active":"2024-04-26T20:26:03.348491+0000","last_peered":"2024-04-26T20:26:03.348491+0000","last_clean":"2024-04-26T20:26:03.348491+0000","last_became_active":"2024-04-26T20:25:58.010833+0000","last_became_peered":"2024-04-26T20:25:58.010833+0000","last_unstale":"2024-04-26T20:26:03.348491+0000","last_undegraded":"2024-04-26T20:26:03.348491+0000","last_fullsized":"2024-04-26T20:26:03.348491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351644+0000","last_change":"2024-04-26T20:25:56.986174+0000","last_active":"2024-04-26T20:26:03.351644+0000","last_peered":"2024-04-26T20:26:03.351644+0000","last_clean":"2024-04-26T20:26:03.351644+0000","last_became_active":"2024-04-26T20:25:56.985869+0000","last_became_peered":"2024-04-26T20:25:56.985869+0000","last_unstale":"2024-04-26T20:26:03.351644+0000","last_undegraded":"2024-04-26T20:26:03.351644+0000","last_fullsized":"2024-04-26T20:26:03.351644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093097+0000","last_change":"2024-04-26T20:26:02.413559+0000","last_active":"2024-04-26T20:26:03.093097+0000","last_peered":"2024-04-26T20:26:03.093097+0000","last_clean":"2024-04-26T20:26:03.093097+0000","last_became_active":"2024-04-26T20:26:02.413096+0000","last_became_peered":"2024-04-26T20:26:02.413096+0000","last_unstale":"2024-04-26T20:26:03.093097+0000","last_undegraded":"2024-04-26T20:26:03.093097+0000","last_fullsized":"2024-04-26T20:26:03.093097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:44.245008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346456+0000","last_change":"2024-04-26T20:26:02.413921+0000","last_active":"2024-04-26T20:26:03.346456+0000","last_peered":"2024-04-26T20:26:03.346456+0000","last_clean":"2024-04-26T20:26:03.346456+0000","last_became_active":"2024-04-26T20:26:02.413388+0000","last_became_peered":"2024-04-26T20:26:02.413388+0000","last_unstale":"2024-04-26T20:26:03.346456+0000","last_undegraded":"2024-04-26T20:26:03.346456+0000","last_fullsized":"2024-04-26T20:26:03.346456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345276+0000","last_change":"2024-04-26T20:25:58.004583+0000","last_active":"2024-04-26T20:26:03.345276+0000","last_peered":"2024-04-26T20:26:03.345276+0000","last_clean":"2024-04-26T20:26:03.345276+0000","last_became_active":"2024-04-26T20:25:58.004228+0000","last_became_peered":"2024-04-26T20:25:58.004228+0000","last_unstale":"2024-04-26T20:26:03.345276+0000","last_undegraded":"2024-04-26T20:26:03.345276+0000","last_fullsized":"2024-04-26T20:26:03.345276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346675+0000","last_change":"2024-04-26T20:25:56.985910+0000","last_active":"2024-04-26T20:26:03.346675+0000","last_peered":"2024-04-26T20:26:03.346675+0000","last_clean":"2024-04-26T20:26:03.346675+0000","last_became_active":"2024-04-26T20:25:56.985636+0000","last_became_peered":"2024-04-26T20:25:56.985636+0000","last_unstale":"2024-04-26T20:26:03.346675+0000","last_undegraded":"2024-04-26T20:26:03.346675+0000","last_fullsized":"2024-04-26T20:26:03.346675+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:02:36.779625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351558+0000","last_change":"2024-04-26T20:26:02.079470+0000","last_active":"2024-04-26T20:26:03.351558+0000","last_peered":"2024-04-26T20:26:03.351558+0000","last_clean":"2024-04-26T20:26:03.351558+0000","last_became_active":"2024-04-26T20:26:02.078853+0000","last_became_peered":"2024-04-26T20:26:02.078853+0000","last_unstale":"2024-04-26T20:26:03.351558+0000","last_undegraded":"2024-04-26T20:26:03.351558+0000","last_fullsized":"2024-04-26T20:26:03.351558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:59.535750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093119+0000","last_change":"2024-04-26T20:25:58.006907+0000","last_active":"2024-04-26T20:26:03.093119+0000","last_peered":"2024-04-26T20:26:03.093119+0000","last_clean":"2024-04-26T20:26:03.093119+0000","last_became_active":"2024-04-26T20:25:58.006370+0000","last_became_peered":"2024-04-26T20:25:58.006370+0000","last_unstale":"2024-04-26T20:26:03.093119+0000","last_undegraded":"2024-04-26T20:26:03.093119+0000","last_fullsized":"2024-04-26T20:26:03.093119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093249+0000","last_change":"2024-04-26T20:25:56.982952+0000","last_active":"2024-04-26T20:26:03.093249+0000","last_peered":"2024-04-26T20:26:03.093249+0000","last_clean":"2024-04-26T20:26:03.093249+0000","last_became_active":"2024-04-26T20:25:56.982240+0000","last_became_peered":"2024-04-26T20:25:56.982240+0000","last_unstale":"2024-04-26T20:26:03.093249+0000","last_undegraded":"2024-04-26T20:26:03.093249+0000","last_fullsized":"2024-04-26T20:26:03.093249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:22.552495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349655+0000","last_change":"2024-04-26T20:25:56.982228+0000","last_active":"2024-04-26T20:26:03.349655+0000","last_peered":"2024-04-26T20:26:03.349655+0000","last_clean":"2024-04-26T20:26:03.349655+0000","last_became_active":"2024-04-26T20:25:56.981083+0000","last_became_peered":"2024-04-26T20:25:56.981083+0000","last_unstale":"2024-04-26T20:26:03.349655+0000","last_undegraded":"2024-04-26T20:26:03.349655+0000","last_fullsized":"2024-04-26T20:26:03.349655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:55.198561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346695+0000","last_change":"2024-04-26T20:25:58.001685+0000","last_active":"2024-04-26T20:26:03.346695+0000","last_peered":"2024-04-26T20:26:03.346695+0000","last_clean":"2024-04-26T20:26:03.346695+0000","last_became_active":"2024-04-26T20:25:58.001160+0000","last_became_peered":"2024-04-26T20:25:58.001160+0000","last_unstale":"2024-04-26T20:26:03.346695+0000","last_undegraded":"2024-04-26T20:26:03.346695+0000","last_fullsized":"2024-04-26T20:26:03.346695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048776+0000","last_change":"2024-04-26T20:26:02.078797+0000","last_active":"2024-04-26T20:26:03.048776+0000","last_peered":"2024-04-26T20:26:03.048776+0000","last_clean":"2024-04-26T20:26:03.048776+0000","last_became_active":"2024-04-26T20:26:02.078444+0000","last_became_peered":"2024-04-26T20:26:02.078444+0000","last_unstale":"2024-04-26T20:26:03.048776+0000","last_undegraded":"2024-04-26T20:26:03.048776+0000","last_fullsized":"2024-04-26T20:26:03.048776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:16.872808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093946+0000","last_change":"2024-04-26T20:25:56.986739+0000","last_active":"2024-04-26T20:26:03.093946+0000","last_peered":"2024-04-26T20:26:03.093946+0000","last_clean":"2024-04-26T20:26:03.093946+0000","last_became_active":"2024-04-26T20:25:56.986399+0000","last_became_peered":"2024-04-26T20:25:56.986399+0000","last_unstale":"2024-04-26T20:26:03.093946+0000","last_undegraded":"2024-04-26T20:26:03.093946+0000","last_fullsized":"2024-04-26T20:26:03.093946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:55.920922+0000","last_clean_scrub_stamp":"2024-04-26T20:25:55.920922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:15.506810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050966+0000","last_change":"2024-04-26T20:25:57.999363+0000","last_active":"2024-04-26T20:26:03.050966+0000","last_peered":"2024-04-26T20:26:03.050966+0000","last_clean":"2024-04-26T20:26:03.050966+0000","last_became_active":"2024-04-26T20:25:57.995297+0000","last_became_peered":"2024-04-26T20:25:57.995297+0000","last_unstale":"2024-04-26T20:26:03.050966+0000","last_undegraded":"2024-04-26T20:26:03.050966+0000","last_fullsized":"2024-04-26T20:26:03.050966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:34.671294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094404+0000","last_change":"2024-04-26T20:26:02.075522+0000","last_active":"2024-04-26T20:26:03.094404+0000","last_peered":"2024-04-26T20:26:03.094404+0000","last_clean":"2024-04-26T20:26:03.094404+0000","last_became_active":"2024-04-26T20:26:02.075089+0000","last_became_peered":"2024-04-26T20:26:02.075089+0000","last_unstale":"2024-04-26T20:26:03.094404+0000","last_undegraded":"2024-04-26T20:26:03.094404+0000","last_fullsized":"2024-04-26T20:26:03.094404+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:26:01.012618+0000","last_clean_scrub_stamp":"2024-04-26T20:26:01.012618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:14:23.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350468+0000","last_change":"2024-04-26T20:25:57.990722+0000","last_active":"2024-04-26T20:26:03.350468+0000","last_peered":"2024-04-26T20:26:03.350468+0000","last_clean":"2024-04-26T20:26:03.350468+0000","last_became_active":"2024-04-26T20:25:57.990382+0000","last_became_peered":"2024-04-26T20:25:57.990382+0000","last_unstale":"2024-04-26T20:26:03.350468+0000","last_undegraded":"2024-04-26T20:26:03.350468+0000","last_fullsized":"2024-04-26T20:26:03.350468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:16.436304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.047754+0000","last_change":"2024-04-26T20:25:58.014226+0000","last_active":"2024-04-26T20:26:03.047754+0000","last_peered":"2024-04-26T20:26:03.047754+0000","last_clean":"2024-04-26T20:26:03.047754+0000","last_became_active":"2024-04-26T20:25:58.013319+0000","last_became_peered":"2024-04-26T20:25:58.013319+0000","last_unstale":"2024-04-26T20:26:03.047754+0000","last_undegraded":"2024-04-26T20:26:03.047754+0000","last_fullsized":"2024-04-26T20:26:03.047754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:29.558152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348155+0000","last_change":"2024-04-26T20:25:58.008173+0000","last_active":"2024-04-26T20:26:03.348155+0000","last_peered":"2024-04-26T20:26:03.348155+0000","last_clean":"2024-04-26T20:26:03.348155+0000","last_became_active":"2024-04-26T20:25:58.007377+0000","last_became_peered":"2024-04-26T20:25:58.007377+0000","last_unstale":"2024-04-26T20:26:03.348155+0000","last_undegraded":"2024-04-26T20:26:03.348155+0000","last_fullsized":"2024-04-26T20:26:03.348155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:51.436725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346409+0000","last_change":"2024-04-26T20:25:58.003184+0000","last_active":"2024-04-26T20:26:03.346409+0000","last_peered":"2024-04-26T20:26:03.346409+0000","last_clean":"2024-04-26T20:26:03.346409+0000","last_became_active":"2024-04-26T20:25:58.002849+0000","last_became_peered":"2024-04-26T20:25:58.002849+0000","last_unstale":"2024-04-26T20:26:03.346409+0000","last_undegraded":"2024-04-26T20:26:03.346409+0000","last_fullsized":"2024-04-26T20:26:03.346409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094328+0000","last_change":"2024-04-26T20:25:57.994419+0000","last_active":"2024-04-26T20:26:03.094328+0000","last_peered":"2024-04-26T20:26:03.094328+0000","last_clean":"2024-04-26T20:26:03.094328+0000","last_became_active":"2024-04-26T20:25:57.994077+0000","last_became_peered":"2024-04-26T20:25:57.994077+0000","last_unstale":"2024-04-26T20:26:03.094328+0000","last_undegraded":"2024-04-26T20:26:03.094328+0000","last_fullsized":"2024-04-26T20:26:03.094328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:35.637794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344289+0000","last_change":"2024-04-26T20:25:58.000446+0000","last_active":"2024-04-26T20:26:03.344289+0000","last_peered":"2024-04-26T20:26:03.344289+0000","last_clean":"2024-04-26T20:26:03.344289+0000","last_became_active":"2024-04-26T20:25:58.000166+0000","last_became_peered":"2024-04-26T20:25:58.000166+0000","last_unstale":"2024-04-26T20:26:03.344289+0000","last_undegraded":"2024-04-26T20:26:03.344289+0000","last_fullsized":"2024-04-26T20:26:03.344289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351690+0000","last_change":"2024-04-26T20:25:57.996288+0000","last_active":"2024-04-26T20:26:03.351690+0000","last_peered":"2024-04-26T20:26:03.351690+0000","last_clean":"2024-04-26T20:26:03.351690+0000","last_became_active":"2024-04-26T20:25:57.995978+0000","last_became_peered":"2024-04-26T20:25:57.995978+0000","last_unstale":"2024-04-26T20:26:03.351690+0000","last_undegraded":"2024-04-26T20:26:03.351690+0000","last_fullsized":"2024-04-26T20:26:03.351690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.094950+0000","last_change":"2024-04-26T20:25:57.993394+0000","last_active":"2024-04-26T20:26:03.094950+0000","last_peered":"2024-04-26T20:26:03.094950+0000","last_clean":"2024-04-26T20:26:03.094950+0000","last_became_active":"2024-04-26T20:25:57.993023+0000","last_became_peered":"2024-04-26T20:25:57.993023+0000","last_unstale":"2024-04-26T20:26:03.094950+0000","last_undegraded":"2024-04-26T20:26:03.094950+0000","last_fullsized":"2024-04-26T20:26:03.094950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346415+0000","last_change":"2024-04-26T20:25:57.997685+0000","last_active":"2024-04-26T20:26:03.346415+0000","last_peered":"2024-04-26T20:26:03.346415+0000","last_clean":"2024-04-26T20:26:03.346415+0000","last_became_active":"2024-04-26T20:25:57.997422+0000","last_became_peered":"2024-04-26T20:25:57.997422+0000","last_unstale":"2024-04-26T20:26:03.346415+0000","last_undegraded":"2024-04-26T20:26:03.346415+0000","last_fullsized":"2024-04-26T20:26:03.346415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093095+0000","last_change":"2024-04-26T20:25:58.003864+0000","last_active":"2024-04-26T20:26:03.093095+0000","last_peered":"2024-04-26T20:26:03.093095+0000","last_clean":"2024-04-26T20:26:03.093095+0000","last_became_active":"2024-04-26T20:25:58.003553+0000","last_became_peered":"2024-04-26T20:25:58.003553+0000","last_unstale":"2024-04-26T20:26:03.093095+0000","last_undegraded":"2024-04-26T20:26:03.093095+0000","last_fullsized":"2024-04-26T20:26:03.093095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347138+0000","last_change":"2024-04-26T20:25:58.007689+0000","last_active":"2024-04-26T20:26:03.347138+0000","last_peered":"2024-04-26T20:26:03.347138+0000","last_clean":"2024-04-26T20:26:03.347138+0000","last_became_active":"2024-04-26T20:25:58.006743+0000","last_became_peered":"2024-04-26T20:25:58.006743+0000","last_unstale":"2024-04-26T20:26:03.347138+0000","last_undegraded":"2024-04-26T20:26:03.347138+0000","last_fullsized":"2024-04-26T20:26:03.347138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.855368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346030+0000","last_change":"2024-04-26T20:25:58.001516+0000","last_active":"2024-04-26T20:26:03.346030+0000","last_peered":"2024-04-26T20:26:03.346030+0000","last_clean":"2024-04-26T20:26:03.346030+0000","last_became_active":"2024-04-26T20:25:58.001241+0000","last_became_peered":"2024-04-26T20:25:58.001241+0000","last_unstale":"2024-04-26T20:26:03.346030+0000","last_undegraded":"2024-04-26T20:26:03.346030+0000","last_fullsized":"2024-04-26T20:26:03.346030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350860+0000","last_change":"2024-04-26T20:25:57.993316+0000","last_active":"2024-04-26T20:26:03.350860+0000","last_peered":"2024-04-26T20:26:03.350860+0000","last_clean":"2024-04-26T20:26:03.350860+0000","last_became_active":"2024-04-26T20:25:57.991497+0000","last_became_peered":"2024-04-26T20:25:57.991497+0000","last_unstale":"2024-04-26T20:26:03.350860+0000","last_undegraded":"2024-04-26T20:26:03.350860+0000","last_fullsized":"2024-04-26T20:26:03.350860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.219271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.348333+0000","last_change":"2024-04-26T20:25:58.004031+0000","last_active":"2024-04-26T20:26:03.348333+0000","last_peered":"2024-04-26T20:26:03.348333+0000","last_clean":"2024-04-26T20:26:03.348333+0000","last_became_active":"2024-04-26T20:25:58.003728+0000","last_became_peered":"2024-04-26T20:25:58.003728+0000","last_unstale":"2024-04-26T20:26:03.348333+0000","last_undegraded":"2024-04-26T20:26:03.348333+0000","last_fullsized":"2024-04-26T20:26:03.348333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:28.610311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.048584+0000","last_change":"2024-04-26T20:25:57.986712+0000","last_active":"2024-04-26T20:26:03.048584+0000","last_peered":"2024-04-26T20:26:03.048584+0000","last_clean":"2024-04-26T20:26:03.048584+0000","last_became_active":"2024-04-26T20:25:57.986233+0000","last_became_peered":"2024-04-26T20:25:57.986233+0000","last_unstale":"2024-04-26T20:26:03.048584+0000","last_undegraded":"2024-04-26T20:26:03.048584+0000","last_fullsized":"2024-04-26T20:26:03.048584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346524+0000","last_change":"2024-04-26T20:25:58.011317+0000","last_active":"2024-04-26T20:26:03.346524+0000","last_peered":"2024-04-26T20:26:03.346524+0000","last_clean":"2024-04-26T20:26:03.346524+0000","last_became_active":"2024-04-26T20:25:58.010881+0000","last_became_peered":"2024-04-26T20:25:58.010881+0000","last_unstale":"2024-04-26T20:26:03.346524+0000","last_undegraded":"2024-04-26T20:26:03.346524+0000","last_fullsized":"2024-04-26T20:26:03.346524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:16:28.165720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.344914+0000","last_change":"2024-04-26T20:25:58.018419+0000","last_active":"2024-04-26T20:26:03.344914+0000","last_peered":"2024-04-26T20:26:03.344914+0000","last_clean":"2024-04-26T20:26:03.344914+0000","last_became_active":"2024-04-26T20:25:58.018134+0000","last_became_peered":"2024-04-26T20:25:58.018134+0000","last_unstale":"2024-04-26T20:26:03.344914+0000","last_undegraded":"2024-04-26T20:26:03.344914+0000","last_fullsized":"2024-04-26T20:26:03.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:45.542062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.343902+0000","last_change":"2024-04-26T20:25:58.008263+0000","last_active":"2024-04-26T20:26:03.343902+0000","last_peered":"2024-04-26T20:26:03.343902+0000","last_clean":"2024-04-26T20:26:03.343902+0000","last_became_active":"2024-04-26T20:25:58.007268+0000","last_became_peered":"2024-04-26T20:25:58.007268+0000","last_unstale":"2024-04-26T20:26:03.343902+0000","last_undegraded":"2024-04-26T20:26:03.343902+0000","last_fullsized":"2024-04-26T20:26:03.343902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345391+0000","last_change":"2024-04-26T20:25:58.011832+0000","last_active":"2024-04-26T20:26:03.345391+0000","last_peered":"2024-04-26T20:26:03.345391+0000","last_clean":"2024-04-26T20:26:03.345391+0000","last_became_active":"2024-04-26T20:25:58.011007+0000","last_became_peered":"2024-04-26T20:25:58.011007+0000","last_unstale":"2024-04-26T20:26:03.345391+0000","last_undegraded":"2024-04-26T20:26:03.345391+0000","last_fullsized":"2024-04-26T20:26:03.345391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:38.909585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.093783+0000","last_change":"2024-04-26T20:25:57.993780+0000","last_active":"2024-04-26T20:26:03.093783+0000","last_peered":"2024-04-26T20:26:03.093783+0000","last_clean":"2024-04-26T20:26:03.093783+0000","last_became_active":"2024-04-26T20:25:57.993444+0000","last_became_peered":"2024-04-26T20:25:57.993444+0000","last_unstale":"2024-04-26T20:26:03.093783+0000","last_undegraded":"2024-04-26T20:26:03.093783+0000","last_fullsized":"2024-04-26T20:26:03.093783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.050252+0000","last_change":"2024-04-26T20:25:57.985098+0000","last_active":"2024-04-26T20:26:03.050252+0000","last_peered":"2024-04-26T20:26:03.050252+0000","last_clean":"2024-04-26T20:26:03.050252+0000","last_became_active":"2024-04-26T20:25:57.984739+0000","last_became_peered":"2024-04-26T20:25:57.984739+0000","last_unstale":"2024-04-26T20:26:03.050252+0000","last_undegraded":"2024-04-26T20:26:03.050252+0000","last_fullsized":"2024-04-26T20:26:03.050252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:13.990600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351223+0000","last_change":"2024-04-26T20:25:58.007442+0000","last_active":"2024-04-26T20:26:03.351223+0000","last_peered":"2024-04-26T20:26:03.351223+0000","last_clean":"2024-04-26T20:26:03.351223+0000","last_became_active":"2024-04-26T20:25:58.007150+0000","last_became_peered":"2024-04-26T20:25:58.007150+0000","last_unstale":"2024-04-26T20:26:03.351223+0000","last_undegraded":"2024-04-26T20:26:03.351223+0000","last_fullsized":"2024-04-26T20:26:03.351223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:10:54.467721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.351686+0000","last_change":"2024-04-26T20:25:58.007696+0000","last_active":"2024-04-26T20:26:03.351686+0000","last_peered":"2024-04-26T20:26:03.351686+0000","last_clean":"2024-04-26T20:26:03.351686+0000","last_became_active":"2024-04-26T20:25:58.007415+0000","last_became_peered":"2024-04-26T20:25:58.007415+0000","last_unstale":"2024-04-26T20:26:03.351686+0000","last_undegraded":"2024-04-26T20:26:03.351686+0000","last_fullsized":"2024-04-26T20:26:03.351686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.350018+0000","last_change":"2024-04-26T20:25:58.014108+0000","last_active":"2024-04-26T20:26:03.350018+0000","last_peered":"2024-04-26T20:26:03.350018+0000","last_clean":"2024-04-26T20:26:03.350018+0000","last_became_active":"2024-04-26T20:25:58.012072+0000","last_became_peered":"2024-04-26T20:25:58.012072+0000","last_unstale":"2024-04-26T20:26:03.350018+0000","last_undegraded":"2024-04-26T20:26:03.350018+0000","last_fullsized":"2024-04-26T20:26:03.350018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:10.380322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.092736+0000","last_change":"2024-04-26T20:25:58.004587+0000","last_active":"2024-04-26T20:26:03.092736+0000","last_peered":"2024-04-26T20:26:03.092736+0000","last_clean":"2024-04-26T20:26:03.092736+0000","last_became_active":"2024-04-26T20:25:58.003068+0000","last_became_peered":"2024-04-26T20:25:58.003068+0000","last_unstale":"2024-04-26T20:26:03.092736+0000","last_undegraded":"2024-04-26T20:26:03.092736+0000","last_fullsized":"2024-04-26T20:26:03.092736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:46.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.345668+0000","last_change":"2024-04-26T20:25:58.017953+0000","last_active":"2024-04-26T20:26:03.345668+0000","last_peered":"2024-04-26T20:26:03.345668+0000","last_clean":"2024-04-26T20:26:03.345668+0000","last_became_active":"2024-04-26T20:25:58.017321+0000","last_became_peered":"2024-04-26T20:25:58.017321+0000","last_unstale":"2024-04-26T20:26:03.345668+0000","last_undegraded":"2024-04-26T20:26:03.345668+0000","last_fullsized":"2024-04-26T20:26:03.345668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:16.489329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346846+0000","last_change":"2024-04-26T20:25:58.007941+0000","last_active":"2024-04-26T20:26:03.346846+0000","last_peered":"2024-04-26T20:26:03.346846+0000","last_clean":"2024-04-26T20:26:03.346846+0000","last_became_active":"2024-04-26T20:25:58.007043+0000","last_became_peered":"2024-04-26T20:25:58.007043+0000","last_unstale":"2024-04-26T20:26:03.346846+0000","last_undegraded":"2024-04-26T20:26:03.346846+0000","last_fullsized":"2024-04-26T20:26:03.346846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:58.821022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.347334+0000","last_change":"2024-04-26T20:25:58.003034+0000","last_active":"2024-04-26T20:26:03.347334+0000","last_peered":"2024-04-26T20:26:03.347334+0000","last_clean":"2024-04-26T20:26:03.347334+0000","last_became_active":"2024-04-26T20:25:58.002792+0000","last_became_peered":"2024-04-26T20:25:58.002792+0000","last_unstale":"2024-04-26T20:26:03.347334+0000","last_undegraded":"2024-04-26T20:26:03.347334+0000","last_fullsized":"2024-04-26T20:26:03.347334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:55.457728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346106+0000","last_change":"2024-04-26T20:25:58.008698+0000","last_active":"2024-04-26T20:26:03.346106+0000","last_peered":"2024-04-26T20:26:03.346106+0000","last_clean":"2024-04-26T20:26:03.346106+0000","last_became_active":"2024-04-26T20:25:58.008279+0000","last_became_peered":"2024-04-26T20:25:58.008279+0000","last_unstale":"2024-04-26T20:26:03.346106+0000","last_undegraded":"2024-04-26T20:26:03.346106+0000","last_fullsized":"2024-04-26T20:26:03.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:55.702652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346355+0000","last_change":"2024-04-26T20:25:57.985508+0000","last_active":"2024-04-26T20:26:03.346355+0000","last_peered":"2024-04-26T20:26:03.346355+0000","last_clean":"2024-04-26T20:26:03.346355+0000","last_became_active":"2024-04-26T20:25:57.985227+0000","last_became_peered":"2024-04-26T20:25:57.985227+0000","last_unstale":"2024-04-26T20:26:03.346355+0000","last_undegraded":"2024-04-26T20:26:03.346355+0000","last_fullsized":"2024-04-26T20:26:03.346355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:16.109722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.346661+0000","last_change":"2024-04-26T20:25:57.989552+0000","last_active":"2024-04-26T20:26:03.346661+0000","last_peered":"2024-04-26T20:26:03.346661+0000","last_clean":"2024-04-26T20:26:03.346661+0000","last_became_active":"2024-04-26T20:25:57.989141+0000","last_became_peered":"2024-04-26T20:25:57.989141+0000","last_unstale":"2024-04-26T20:26:03.346661+0000","last_undegraded":"2024-04-26T20:26:03.346661+0000","last_fullsized":"2024-04-26T20:26:03.346661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:10.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-26T20:26:03.349524+0000","last_change":"2024-04-26T20:25:58.003504+0000","last_active":"2024-04-26T20:26:03.349524+0000","last_peered":"2024-04-26T20:26:03.349524+0000","last_clean":"2024-04-26T20:26:03.349524+0000","last_became_active":"2024-04-26T20:25:58.002880+0000","last_became_peered":"2024-04-26T20:25:58.002880+0000","last_unstale":"2024-04-26T20:26:03.349524+0000","last_undegraded":"2024-04-26T20:26:03.349524+0000","last_fullsized":"2024-04-26T20:26:03.349524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:56.934596+0000","last_clean_scrub_stamp":"2024-04-26T20:25:56.934596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:47.751608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:26:32.819 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:26:32.819 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T20:26:32.819 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T20:26:32.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:26:33.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.062+0000 7fb097e00640 1 Processor -- start 2024-04-26T20:26:33.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.062+0000 7fb097e00640 1 -- start start 2024-04-26T20:26:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb098161460 0x7fb098160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb098160d90 0x7fb09806e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0980b50e0 con 0x7fb0981463e0 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb0981611d0 con 0x7fb098161460 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb0980b4560 con 0x7fb098160d90 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb096000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb098161460 0x7fb098160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb096000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb098161460 0x7fb098160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54926/0 (socket says 172.21.15.43:54926) 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb096000640 1 -- 172.21.15.43:0/1914855374 learned_addr learned my addr 172.21.15.43:0/1914855374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:33.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.064+0000 7fb097400640 1 --2- 172.21.15.43:0/1914855374 >> v2:172.21.15.164:3300/0 conn(0x7fb098160d90 0x7fb09806e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.065+0000 7fb096000640 1 -- 172.21.15.43:0/1914855374 >> v2:172.21.15.164:3300/0 conn(0x7fb098160d90 msgr2=0x7fb09806e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:33.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.065+0000 7fb096000640 1 --2- 172.21.15.43:0/1914855374 >> v2:172.21.15.164:3300/0 conn(0x7fb098160d90 0x7fb09806e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.065+0000 7fb096000640 1 -- 172.21.15.43:0/1914855374 >> v1:172.21.15.164:6789/0 conn(0x7fb0981463e0 legacy=0x7fb0980b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.065+0000 7fb096000640 1 -- 172.21.15.43:0/1914855374 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0980b4840 con 0x7fb098161460 2024-04-26T20:26:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.066+0000 7fb096000640 1 --2- 172.21.15.43:0/1914855374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb098160850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb09806d670 tx=0x7fb08802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f843fee88f1c7980 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.066+0000 7fb095600640 1 -- 172.21.15.43:0/1914855374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088036070 con 0x7fb098161460 2024-04-26T20:26:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.066+0000 7fb095600640 1 -- 172.21.15.43:0/1914855374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb088031600 con 0x7fb098161460 2024-04-26T20:26:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.066+0000 7fb095600640 1 -- 172.21.15.43:0/1914855374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880319d0 con 0x7fb098161460 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 -- 172.21.15.43:0/1914855374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 msgr2=0x7fb098160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 --2- 172.21.15.43:0/1914855374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb098160850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb09806d670 tx=0x7fb08802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 -- 172.21.15.43:0/1914855374 shutdown_connections 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 --2- 172.21.15.43:0/1914855374 >> v2:172.21.15.164:3300/0 conn(0x7fb098160d90 0x7fb09806e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 --2- 172.21.15.43:0/1914855374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb098160850 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 -- 172.21.15.43:0/1914855374 >> 172.21.15.43:0/1914855374 conn(0x7fb0980b26b0 msgr2=0x7fb0980b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 -- 172.21.15.43:0/1914855374 shutdown_connections 2024-04-26T20:26:33.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.067+0000 7fb097e00640 1 -- 172.21.15.43:0/1914855374 wait complete. 2024-04-26T20:26:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.069+0000 7fb097e00640 1 Processor -- start 2024-04-26T20:26:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.069+0000 7fb097e00640 1 -- start start 2024-04-26T20:26:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0981463e0 0x7fb09813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 0x7fb098072690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981467c0 con 0x7fb098161460 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0980b4e00 con 0x7fb098160d90 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb09806e590 con 0x7fb0981463e0 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.070+0000 7fb097400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54934/0 (socket says 172.21.15.43:54934) 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 -- 172.21.15.43:0/632496048 learned_addr learned my addr 172.21.15.43:0/632496048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb096000640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 0x7fb098072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb096a00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0981463e0 0x7fb09813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 -- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0981463e0 msgr2=0x7fb09813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0981463e0 0x7fb09813c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 -- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 msgr2=0x7fb098072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 0x7fb098072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 -- 172.21.15.43:0/632496048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09813c9a0 con 0x7fb098161460 2024-04-26T20:26:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb096000640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 0x7fb098072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.071+0000 7fb097400640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e960 tx=0x7fb09000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.072+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.072+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09000c980 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.072+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000cd70 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.072+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09813d000 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.072+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb098073250 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.073+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09813d1d0 con 0x7fb098161460 2024-04-26T20:26:33.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.074+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb09001a070 con 0x7fb098161460 2024-04-26T20:26:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.075+0000 7fb08ea00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 0x7fb07406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.075+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb09008be40 con 0x7fb098161460 2024-04-26T20:26:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.075+0000 7fb096a00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 0x7fb07406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.076+0000 7fb096a00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 0x7fb07406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.080+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb09813c9a0 con 0x7fb098161460 2024-04-26T20:26:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.319+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb098073f40 con 0x7fb098161460 2024-04-26T20:26:33.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.320+0000 7fb08ea00640 1 -- 172.21.15.43:0/632496048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb090059f20 con 0x7fb098161460 2024-04-26T20:26:33.321 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:33.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:26:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.322+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 msgr2=0x7fb07406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.322+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 0x7fb07406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.322+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 msgr2=0x7fb0980b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.322+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e960 tx=0x7fb09000ee20 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 shutdown_connections 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb07406c8f0 0x7fb07406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb098161460 0x7fb0980b0af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb098160d90 0x7fb098072690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 --2- 172.21.15.43:0/632496048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0981463e0 0x7fb09813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 >> 172.21.15.43:0/632496048 conn(0x7fb0980b26b0 msgr2=0x7fb0980b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 shutdown_connections 2024-04-26T20:26:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.323+0000 7fb097e00640 1 -- 172.21.15.43:0/632496048 wait complete. 2024-04-26T20:26:33.343 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T20:26:33.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.547+0000 7f3dd0a00640 1 Processor -- start 2024-04-26T20:26:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.547+0000 7f3dd0a00640 1 -- start start 2024-04-26T20:26:33.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dd0a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dcc0ce480 0x7f3dcc167b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dd0a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc072a80 con 0x7f3dcc0cd180 2024-04-26T20:26:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dd0a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc0cd560 con 0x7f3dcc072620 2024-04-26T20:26:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dd0a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc072d00 con 0x7f3dcc0ce480 2024-04-26T20:26:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dcb400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3dcc0cd180 0x7f3dcc15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41570/0 (socket says 172.21.15.43:41570) 2024-04-26T20:26:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dcb400640 1 -- 172.21.15.43:0/891647298 learned_addr learned my addr 172.21.15.43:0/891647298 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.549+0000 7f3dcbe00640 1 --2- 172.21.15.43:0/891647298 >> v2:172.21.15.164:3300/0 conn(0x7f3dcc0ce480 0x7f3dcc167b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342650763 0 0) 0x7f3dbc001110 con 0x7f3dcc0cd180 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3db0003520 con 0x7f3dcc0cd180 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 452789033 0 0) 0x7f3db0003520 con 0x7f3dcc0cd180 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 >> v2:172.21.15.164:3300/0 conn(0x7f3dcc0ce480 msgr2=0x7f3dcc167b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 --2- 172.21.15.43:0/891647298 >> v2:172.21.15.164:3300/0 conn(0x7f3dcc0ce480 0x7f3dcc167b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 >> v1:172.21.15.164:6789/0 conn(0x7f3dcc072620 legacy=0x7f3dcc0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc072fe0 con 0x7f3dcc0cd180 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f3dbc001840 con 0x7f3dcc0cd180 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 >> v1:172.21.15.43:6789/0 conn(0x7f3dcc0cd180 legacy=0x7f3dcc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db0004800 0x7f3db0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00051c0 0x7f3db000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc072d00 con 0x7f3db00039b0 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3db000c060 con 0x7f3db0004800 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3db000c260 con 0x7f3db00051c0 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.550+0000 7f3dcb400640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcb400640 1 -- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00051c0 msgr2=0x7f3db000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcb400640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00051c0 0x7f3db000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcb400640 1 -- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db0004800 msgr2=0x7f3db0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcb400640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db0004800 0x7f3db0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcb400640 1 -- 172.21.15.43:0/891647298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db0003520 con 0x7f3db00039b0 2024-04-26T20:26:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.551+0000 7f3dcbe00640 1 -- 172.21.15.43:0/891647298 reap_dead start 2024-04-26T20:26:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.552+0000 7f3dcb400640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c4c0 tx=0x7f3db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475c7ff8f63ab8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.552+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00c0e0 con 0x7f3db00039b0 2024-04-26T20:26:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.552+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc034070 con 0x7f3db00039b0 2024-04-26T20:26:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.552+0000 7f3dca000640 1 -- 172.21.15.43:0/891647298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc005db0 con 0x7f3db00039b0 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 -- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 msgr2=0x7f3db0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c4c0 tx=0x7f3db0005830 comp rx=0 tx=0).stop 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 -- 172.21.15.43:0/891647298 shutdown_connections 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/891647298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db00039b0 0x7f3db0003dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 -- 172.21.15.43:0/891647298 >> 172.21.15.43:0/891647298 conn(0x7f3dcc068090 msgr2=0x7f3dcc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 -- 172.21.15.43:0/891647298 shutdown_connections 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.553+0000 7f3dd0a00640 1 -- 172.21.15.43:0/891647298 wait complete. 2024-04-26T20:26:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.555+0000 7f3dd0a00640 1 Processor -- start 2024-04-26T20:26:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.555+0000 7f3dd0a00640 1 -- start start 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00039b0 0x7f3dcc1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 0x7f3dcc0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc073da0 con 0x7f3dcc1445a0 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc0735a0 con 0x7f3dcc0c8020 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dd0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc0cdd70 con 0x7f3db00039b0 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54946/0 (socket says 172.21.15.43:54946) 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dcbe00640 1 -- 172.21.15.43:0/3513177437 learned_addr learned my addr 172.21.15.43:0/3513177437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dcb400640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00039b0 0x7f3dcc1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.556+0000 7f3dcaa00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 0x7f3dcc0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 -- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00039b0 msgr2=0x7f3dcc1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00039b0 0x7f3dcc1f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 -- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 msgr2=0x7f3dcc0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 0x7f3dcc0c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 -- 172.21.15.43:0/3513177437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc0c5950 con 0x7f3dcc1445a0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcaa00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 0x7f3dcc0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dcbe00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eaa0 tx=0x7f3dc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400bbd0 con 0x7f3dcc1445a0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc401a070 con 0x7f3dcc1445a0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4014400 con 0x7f3dcc1445a0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1452d0 con 0x7f3dcc1445a0 2024-04-26T20:26:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.557+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc0c5ea0 con 0x7f3dcc1445a0 2024-04-26T20:26:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.558+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc0cd790 con 0x7f3dcc1445a0 2024-04-26T20:26:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.559+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dc4005d10 con 0x7f3dcc1445a0 2024-04-26T20:26:33.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.560+0000 7f3dc3400640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:33.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.560+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3dc408bf20 con 0x7f3dcc1445a0 2024-04-26T20:26:33.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.561+0000 7f3dcb400640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:33.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.561+0000 7f3dcb400640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 0x7f3dac06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c490 tx=0x7f3dbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:33.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.565+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3dcc0cd790 con 0x7f3dcc1445a0 2024-04-26T20:26:33.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.772+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dcc069350 con 0x7f3dcc1445a0 2024-04-26T20:26:33.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.775+0000 7f3dc3400640 1 -- 172.21.15.43:0/3513177437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f3dc4059f70 con 0x7f3dcc1445a0 2024-04-26T20:26:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:33.777 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:33.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 msgr2=0x7f3dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 0x7f3dac06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c490 tx=0x7f3dbc02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 msgr2=0x7f3dcc144a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eaa0 tx=0x7f3dc400ef60 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 shutdown_connections 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dcc1445a0 0x7f3dcc144a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dcc0c8020 0x7f3dcc0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 --2- 172.21.15.43:0/3513177437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3db00039b0 0x7f3dcc1f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 >> 172.21.15.43:0/3513177437 conn(0x7f3dcc068090 msgr2=0x7f3dcc142340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 shutdown_connections 2024-04-26T20:26:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.778+0000 7f3dd0a00640 1 -- 172.21.15.43:0/3513177437 wait complete. 2024-04-26T20:26:33.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:33.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:33.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.997+0000 7f9831200640 1 Processor -- start 2024-04-26T20:26:33.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.997+0000 7f9831200640 1 -- start start 2024-04-26T20:26:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.998+0000 7f9831200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.998+0000 7f9831200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f982c0d3830 0x7f982c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.998+0000 7f9831200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c0d2d70 con 0x7f982c0b3cb0 2024-04-26T20:26:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.998+0000 7f9831200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f982c15f560 con 0x7f982c0d3830 2024-04-26T20:26:34.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.998+0000 7f9831200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0d06e0 con 0x7f982c15f180 2024-04-26T20:26:34.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.999+0000 7f982be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f982c0d3830 0x7f982c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.999+0000 7f982b400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.999+0000 7f982be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f982c0d3830 0x7f982c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:46846/0 (socket says 172.21.15.43:46846) 2024-04-26T20:26:34.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:33.999+0000 7f982be00640 1 -- 172.21.15.43:0/2040512622 learned_addr learned my addr 172.21.15.43:0/2040512622 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.000+0000 7f982be00640 1 -- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 msgr2=0x7f982c0d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.000+0000 7f982be00640 1 --2- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.000+0000 7f982be00640 1 -- 172.21.15.43:0/2040512622 >> v1:172.21.15.164:6789/0 conn(0x7f982c0b3cb0 legacy=0x7f982c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.000+0000 7f982be00640 1 -- 172.21.15.43:0/2040512622 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0d09c0 con 0x7f982c0d3830 2024-04-26T20:26:34.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.001+0000 7f982be00640 1 --2- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0d3830 0x7f982c0b1740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9824009870 tx=0x7f982402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a44b31f3f874f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f982a000640 1 -- 172.21.15.43:0/2040512622 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824037070 con 0x7f982c0d3830 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f982a000640 1 -- 172.21.15.43:0/2040512622 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9824032440 con 0x7f982c0d3830 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f982a000640 1 -- 172.21.15.43:0/2040512622 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824032810 con 0x7f982c0d3830 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f9831200640 1 -- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0d3830 msgr2=0x7f982c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f9831200640 1 --2- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0d3830 0x7f982c0b1740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9824009870 tx=0x7f982402a270 comp rx=0 tx=0).stop 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f9831200640 1 -- 172.21.15.43:0/2040512622 shutdown_connections 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f9831200640 1 --2- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0d3830 0x7f982c0b1740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.002+0000 7f9831200640 1 --2- 172.21.15.43:0/2040512622 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.003+0000 7f9831200640 1 -- 172.21.15.43:0/2040512622 >> 172.21.15.43:0/2040512622 conn(0x7f982c0d4bd0 msgr2=0x7f982c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.003+0000 7f9831200640 1 -- 172.21.15.43:0/2040512622 shutdown_connections 2024-04-26T20:26:34.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.003+0000 7f9831200640 1 -- 172.21.15.43:0/2040512622 wait complete. 2024-04-26T20:26:34.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.004+0000 7f9831200640 1 Processor -- start 2024-04-26T20:26:34.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.004+0000 7f9831200640 1 -- start start 2024-04-26T20:26:34.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0b3cb0 0x7f982c152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0d11c0 con 0x7f982c0d3830 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0d2a40 con 0x7f982c0b3cb0 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.005+0000 7f9831200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f982c153770 con 0x7f982c15f180 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54948/0 (socket says 172.21.15.43:54948) 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 -- 172.21.15.43:0/3459311699 learned_addr learned my addr 172.21.15.43:0/3459311699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982b400640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0b3cb0 0x7f982c152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982be00640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 -- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 msgr2=0x7f982c0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 -- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0b3cb0 msgr2=0x7f982c152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0b3cb0 0x7f982c152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.006+0000 7f982aa00640 1 -- 172.21.15.43:0/3459311699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0b7a10 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f982aa00640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f981400c7a0 tx=0x7f981400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814016020 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9814006d50 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814003bb0 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c0b7cf0 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.007+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c0b82b0 con 0x7f982c0d3830 2024-04-26T20:26:34.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.008+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0506c0 con 0x7f982c0d3830 2024-04-26T20:26:34.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.009+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f981400a850 con 0x7f982c0d3830 2024-04-26T20:26:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.011+0000 7f9823400640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 0x7f980806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.011+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f981408bbd0 con 0x7f982c0d3830 2024-04-26T20:26:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.011+0000 7f982b400640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 0x7f980806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.012+0000 7f982b400640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 0x7f980806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f981c009a30 tx=0x7f981c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.016+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9814020020 con 0x7f982c0d3830 2024-04-26T20:26:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.228+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f982c0d3610 con 0x7f982c0d3830 2024-04-26T20:26:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.229+0000 7f9823400640 1 -- 172.21.15.43:0/3459311699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f9814019310 con 0x7f982c0d3830 2024-04-26T20:26:34.231 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 msgr2=0x7f980806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 0x7f980806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f981c009a30 tx=0x7f981c008040 comp rx=0 tx=0).stop 2024-04-26T20:26:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 msgr2=0x7f982c0cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f981400c7a0 tx=0x7f981400cc60 comp rx=0 tx=0).stop 2024-04-26T20:26:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 shutdown_connections 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f980806c8f0 0x7f980806edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f982c15f180 0x7f982c0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f982c0d3830 0x7f982c0cdb80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 --2- 172.21.15.43:0/3459311699 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f982c0b3cb0 0x7f982c152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.232+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 >> 172.21.15.43:0/3459311699 conn(0x7f982c0d4bd0 msgr2=0x7f982c0b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.233+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 shutdown_connections 2024-04-26T20:26:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.233+0000 7f9831200640 1 -- 172.21.15.43:0/3459311699 wait complete. 2024-04-26T20:26:34.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.469+0000 7f19ae200640 1 Processor -- start 2024-04-26T20:26:34.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.469+0000 7f19ae200640 1 -- start start 2024-04-26T20:26:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.470+0000 7f19ae200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19a815a7e0 0x7f19a80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ae200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f19a80b5410 0x7f19a80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ae200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a80b3900 con 0x7f19a815d2b0 2024-04-26T20:26:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ae200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f19a815abc0 con 0x7f19a815a7e0 2024-04-26T20:26:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ae200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f19a80b3b70 con 0x7f19a80b5410 2024-04-26T20:26:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19a815a7e0 0x7f19a80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19a815a7e0 0x7f19a80b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54956/0 (socket says 172.21.15.43:54956) 2024-04-26T20:26:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 -- 172.21.15.43:0/696731029 learned_addr learned my addr 172.21.15.43:0/696731029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19a7e00640 1 --2- 172.21.15.43:0/696731029 >> v2:172.21.15.164:3300/0 conn(0x7f19a80b5410 0x7f19a80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 -- 172.21.15.43:0/696731029 >> v2:172.21.15.164:3300/0 conn(0x7f19a80b5410 msgr2=0x7f19a80b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 --2- 172.21.15.43:0/696731029 >> v2:172.21.15.164:3300/0 conn(0x7f19a80b5410 0x7f19a80b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.471+0000 7f19ace00640 1 -- 172.21.15.43:0/696731029 >> v1:172.21.15.164:6789/0 conn(0x7f19a815d2b0 legacy=0x7f19a815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.472+0000 7f19ace00640 1 -- 172.21.15.43:0/696731029 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a80b3e50 con 0x7f19a815a7e0 2024-04-26T20:26:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.473+0000 7f19ace00640 1 --2- 172.21.15.43:0/696731029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a80b4650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19a80d15f0 tx=0x7f199c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a1e8baf2e65163 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.473+0000 7f19a7400640 1 -- 172.21.15.43:0/696731029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c037070 con 0x7f19a815a7e0 2024-04-26T20:26:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.473+0000 7f19a7400640 1 -- 172.21.15.43:0/696731029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c032440 con 0x7f19a815a7e0 2024-04-26T20:26:34.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.473+0000 7f19a7400640 1 -- 172.21.15.43:0/696731029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c032810 con 0x7f19a815a7e0 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 -- 172.21.15.43:0/696731029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 msgr2=0x7f19a80b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 --2- 172.21.15.43:0/696731029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a80b4650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19a80d15f0 tx=0x7f199c02a230 comp rx=0 tx=0).stop 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 -- 172.21.15.43:0/696731029 shutdown_connections 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 --2- 172.21.15.43:0/696731029 >> v2:172.21.15.164:3300/0 conn(0x7f19a80b5410 0x7f19a80b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 --2- 172.21.15.43:0/696731029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a80b4650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 -- 172.21.15.43:0/696731029 >> 172.21.15.43:0/696731029 conn(0x7f19a8068090 msgr2=0x7f19a80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 -- 172.21.15.43:0/696731029 shutdown_connections 2024-04-26T20:26:34.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.474+0000 7f19ae200640 1 -- 172.21.15.43:0/696731029 wait complete. 2024-04-26T20:26:34.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.476+0000 7f19ae200640 1 Processor -- start 2024-04-26T20:26:34.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.476+0000 7f19ae200640 1 -- start start 2024-04-26T20:26:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19a80b5410 0x7f19a80adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19a815d2b0 0x7f19a8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80b4b90 con 0x7f19a815a7e0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19a815e010 con 0x7f19a815d2b0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19ae200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f19a815e2e0 con 0x7f19a80b5410 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54970/0 (socket says 172.21.15.43:54970) 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.477+0000 7f19a7e00640 1 -- 172.21.15.43:0/2399147494 learned_addr learned my addr 172.21.15.43:0/2399147494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19ace00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19a80b5410 0x7f19a80adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19ad800640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19a815d2b0 0x7f19a8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 -- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19a80b5410 msgr2=0x7f19a80adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19a80b5410 0x7f19a80adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 -- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19a815d2b0 msgr2=0x7f19a8158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19a815d2b0 0x7f19a8158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 -- 172.21.15.43:0/2399147494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a81e8c20 con 0x7f19a815a7e0 2024-04-26T20:26:34.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a7e00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f199400d800 tx=0x7f199400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.478+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400b6c0 con 0x7f19a815a7e0 2024-04-26T20:26:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.479+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1994019070 con 0x7f19a815a7e0 2024-04-26T20:26:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.479+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400ba80 con 0x7f19a815a7e0 2024-04-26T20:26:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.479+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a81e8f00 con 0x7f19a815a7e0 2024-04-26T20:26:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.479+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a80ae9d0 con 0x7f19a815a7e0 2024-04-26T20:26:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.480+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a80506c0 con 0x7f19a815a7e0 2024-04-26T20:26:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.481+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f199400bc20 con 0x7f19a815a7e0 2024-04-26T20:26:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.482+0000 7f19a4c00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 0x7f198806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.482+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f199408c420 con 0x7f19a815a7e0 2024-04-26T20:26:34.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.482+0000 7f19ace00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 0x7f198806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.483+0000 7f19ace00640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 0x7f198806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f199c034000 tx=0x7f199c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.487+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f199401c360 con 0x7f19a815a7e0 2024-04-26T20:26:34.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.690+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19a815f5d0 con 0x7f19a815a7e0 2024-04-26T20:26:34.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.691+0000 7f19a4c00640 1 -- 172.21.15.43:0/2399147494 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f199405a470 con 0x7f19a815a7e0 2024-04-26T20:26:34.693 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 msgr2=0x7f198806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 0x7f198806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f199c034000 tx=0x7f199c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 msgr2=0x7f19a81e81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f199400d800 tx=0x7f199400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 shutdown_connections 2024-04-26T20:26:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198806c8f0 0x7f198806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19a815d2b0 0x7f19a8158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19a815a7e0 0x7f19a81e81d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 --2- 172.21.15.43:0/2399147494 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19a80b5410 0x7f19a80adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.694+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 >> 172.21.15.43:0/2399147494 conn(0x7f19a8068090 msgr2=0x7f19a80cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.695+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 shutdown_connections 2024-04-26T20:26:34.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.695+0000 7f19ae200640 1 -- 172.21.15.43:0/2399147494 wait complete. 2024-04-26T20:26:34.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.933+0000 7f3012a00640 1 Processor -- start 2024-04-26T20:26:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.933+0000 7f3012a00640 1 -- start start 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3012a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f300c0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3012a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f300c15d5e0 0x7f300c0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3012a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c146160 con 0x7f300c145d80 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3012a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f300c06dfb0 con 0x7f300c15d5e0 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3012a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f300c074180 con 0x7f300c072a90 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f300be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f300c145d80 0x7f300c072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41586/0 (socket says 172.21.15.43:41586) 2024-04-26T20:26:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f300be00640 1 -- 172.21.15.43:0/2114407006 learned_addr learned my addr 172.21.15.43:0/2114407006 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f300b400640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f300c0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f3010a00640 1 --2- 172.21.15.43:0/2114407006 >> v2:172.21.15.164:3300/0 conn(0x7f300c15d5e0 0x7f300c0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 243858492 0 0) 0x7f300c146160 con 0x7f300c145d80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.935+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ff0003520 con 0x7f300c145d80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 972820808 0 0) 0x7f2ff0003520 con 0x7f300c145d80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 msgr2=0x7f300c0cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f300c0cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 >> v2:172.21.15.164:3300/0 conn(0x7f300c15d5e0 msgr2=0x7f300c0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 --2- 172.21.15.43:0/2114407006 >> v2:172.21.15.164:3300/0 conn(0x7f300c15d5e0 0x7f300c0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c0d0710 con 0x7f300c145d80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f2ffc0019f0 con 0x7f300c145d80 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 >> v1:172.21.15.43:6789/0 conn(0x7f300c145d80 legacy=0x7f300c072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff00039b0 0x7f2ff0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f2ff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff0004d40 0x7f2ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f300c06dfb0 con 0x7f2ff00039b0 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f300c074180 con 0x7f2ff0004d40 2024-04-26T20:26:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff000b9c0 con 0x7f300c072a90 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.936+0000 7f300be00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff00039b0 0x7f2ff0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300b400640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f2ff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f3010a00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff0004d40 0x7f2ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300be00640 1 -- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 msgr2=0x7f2ff0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300be00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c072a90 0x7f2ff0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300be00640 1 -- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff0004d40 msgr2=0x7f2ff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300be00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff0004d40 0x7f2ff00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.937+0000 7f300be00640 1 -- 172.21.15.43:0/2114407006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff0003520 con 0x7f2ff00039b0 2024-04-26T20:26:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.938+0000 7f300be00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff00039b0 0x7f2ff0003dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00a8a0 tx=0x7f2ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a424f1e6da7995b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.938+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00a4c0 con 0x7f2ff00039b0 2024-04-26T20:26:34.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc02f070 con 0x7f2ff00039b0 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f300aa00640 1 -- 172.21.15.43:0/2114407006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc02bcd0 con 0x7f2ff00039b0 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 -- 172.21.15.43:0/2114407006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff00039b0 msgr2=0x7f2ff0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 --2- 172.21.15.43:0/2114407006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff00039b0 0x7f2ff0003dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00a8a0 tx=0x7f2ff0005830 comp rx=0 tx=0).stop 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3010a00640 1 -- 172.21.15.43:0/2114407006 reap_dead start 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 -- 172.21.15.43:0/2114407006 shutdown_connections 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 -- 172.21.15.43:0/2114407006 >> 172.21.15.43:0/2114407006 conn(0x7f300c0b2900 msgr2=0x7f300c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 -- 172.21.15.43:0/2114407006 shutdown_connections 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.939+0000 7f3012a00640 1 -- 172.21.15.43:0/2114407006 wait complete. 2024-04-26T20:26:34.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.941+0000 7f3012a00640 1 Processor -- start 2024-04-26T20:26:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.941+0000 7f3012a00640 1 -- start start 2024-04-26T20:26:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff00039b0 0x7f300c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 0x7f300c0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0d09f0 con 0x7f300c1f4a70 2024-04-26T20:26:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f300c06e530 con 0x7f2ff00039b0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.942+0000 7f3012a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f300c1468c0 con 0x7f300c1f4ef0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54994/0 (socket says 172.21.15.43:54994) 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 -- 172.21.15.43:0/3240241255 learned_addr learned my addr 172.21.15.43:0/3240241255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300be00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff00039b0 0x7f300c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f3010a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 0x7f300c0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 -- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 msgr2=0x7f300c0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 0x7f300c0af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 -- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff00039b0 msgr2=0x7f300c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff00039b0 0x7f300c15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f300b400640 1 -- 172.21.15.43:0/3240241255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c0aa700 con 0x7f300c1f4a70 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.943+0000 7f3010a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 0x7f300c0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f300b400640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4011600 tx=0x7f2ff4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400f920 con 0x7f300c1f4a70 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff400fac0 con 0x7f300c1f4a70 2024-04-26T20:26:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400fe70 con 0x7f300c1f4a70 2024-04-26T20:26:34.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c0aa9e0 con 0x7f300c1f4a70 2024-04-26T20:26:34.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.944+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c15bc10 con 0x7f300c1f4a70 2024-04-26T20:26:34.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.945+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c069350 con 0x7f300c1f4a70 2024-04-26T20:26:34.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.946+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ff4015460 con 0x7f300c1f4a70 2024-04-26T20:26:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.947+0000 7f3003e00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 0x7f2fe806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.947+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ff408c800 con 0x7f300c1f4a70 2024-04-26T20:26:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.948+0000 7f300be00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 0x7f2fe806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:34.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.948+0000 7f300be00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 0x7f2fe806ee00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033000 tx=0x7f2ffc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:34.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:34.951+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ff4019030 con 0x7f300c1f4a70 2024-04-26T20:26:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.159+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f300c0b2440 con 0x7f300c1f4a70 2024-04-26T20:26:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.160+0000 7f3003e00640 1 -- 172.21.15.43:0/3240241255 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2ff405a960 con 0x7f300c1f4a70 2024-04-26T20:26:35.161 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.162+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 msgr2=0x7f2fe806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.162+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 0x7f2fe806ee00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033000 tx=0x7f2ffc007000 comp rx=0 tx=0).stop 2024-04-26T20:26:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.162+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 msgr2=0x7f300c0aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.162+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4011600 tx=0x7f2ff4011ac0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 shutdown_connections 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2fe806c940 0x7f2fe806ee00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f300c1f4ef0 0x7f300c0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c1f4a70 0x7f300c0aeb00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 --2- 172.21.15.43:0/3240241255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ff00039b0 0x7f300c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 >> 172.21.15.43:0/3240241255 conn(0x7f300c0b2900 msgr2=0x7f300c0cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 shutdown_connections 2024-04-26T20:26:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.163+0000 7f3012a00640 1 -- 172.21.15.43:0/3240241255 wait complete. 2024-04-26T20:26:35.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:35.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.401+0000 7f5ac9a00640 1 Processor -- start 2024-04-26T20:26:35.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.401+0000 7f5ac9a00640 1 -- start start 2024-04-26T20:26:35.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.403+0000 7f5ac9a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ac4155cd0 0x7f5ac4072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.403+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac40b5b10 0x7f5ac4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.403+0000 7f5ac9a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac415f920 con 0x7f5ac4072b40 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.403+0000 7f5ac9a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac415fc00 con 0x7f5ac4155cd0 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.403+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac415fee0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ac4155cd0 0x7f5ac4072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5abbe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5ac4072b40 0x7f5ac40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41592/0 (socket says 172.21.15.43:41592) 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5abbe00640 1 -- 172.21.15.43:0/2953762030 learned_addr learned my addr 172.21.15.43:0/2953762030 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac9000640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac40b5b10 0x7f5ac4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3400640 1 -- 172.21.15.43:0/2953762030 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807513025 0 0) 0x7f5ac415f920 con 0x7f5ac4072b40 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3400640 1 -- 172.21.15.43:0/2953762030 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aa8003520 con 0x7f5ac4072b40 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3e00640 1 -- 172.21.15.43:0/2953762030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac40b5b10 msgr2=0x7f5ac4162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3e00640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac40b5b10 0x7f5ac4162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3e00640 1 -- 172.21.15.43:0/2953762030 >> v1:172.21.15.43:6789/0 conn(0x7f5ac4072b40 legacy=0x7f5ac40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.404+0000 7f5ac3e00640 1 -- 172.21.15.43:0/2953762030 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac41601c0 con 0x7f5ac4155cd0 2024-04-26T20:26:35.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac3e00640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155cd0 0x7f5ac4072600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40097a0 tx=0x7f5ab402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61660a575f05e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac3400640 1 -- 172.21.15.43:0/2953762030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037070 con 0x7f5ac4155cd0 2024-04-26T20:26:35.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac3400640 1 -- 172.21.15.43:0/2953762030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab4032440 con 0x7f5ac4155cd0 2024-04-26T20:26:35.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac3400640 1 -- 172.21.15.43:0/2953762030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40327f0 con 0x7f5ac4155cd0 2024-04-26T20:26:35.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac9a00640 1 -- 172.21.15.43:0/2953762030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155cd0 msgr2=0x7f5ac4072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.406+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155cd0 0x7f5ac4072600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40097a0 tx=0x7f5ab402a210 comp rx=0 tx=0).stop 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 -- 172.21.15.43:0/2953762030 shutdown_connections 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac40b5b10 0x7f5ac4162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/2953762030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac4155cd0 0x7f5ac4072600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 -- 172.21.15.43:0/2953762030 >> 172.21.15.43:0/2953762030 conn(0x7f5ac4068090 msgr2=0x7f5ac40684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 -- 172.21.15.43:0/2953762030 shutdown_connections 2024-04-26T20:26:35.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.407+0000 7f5ac9a00640 1 -- 172.21.15.43:0/2953762030 wait complete. 2024-04-26T20:26:35.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.409+0000 7f5ac9a00640 1 Processor -- start 2024-04-26T20:26:35.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.409+0000 7f5ac9a00640 1 -- start start 2024-04-26T20:26:35.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac4072b40 0x7f5ac415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 0x7f5ac40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4156230 con 0x7f5ac40b5b10 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41560b0 con 0x7f5ac4072b40 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac415f3f0 con 0x7f5ac4155cd0 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5abbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5abbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55026/0 (socket says 172.21.15.43:55026) 2024-04-26T20:26:35.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5abbe00640 1 -- 172.21.15.43:0/1690350717 learned_addr learned my addr 172.21.15.43:0/1690350717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac3e00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac4072b40 0x7f5ac415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.410+0000 7f5ac9000640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 0x7f5ac40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 -- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 msgr2=0x7f5ac40b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 0x7f5ac40b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 -- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac4072b40 msgr2=0x7f5ac415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac4072b40 0x7f5ac415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 -- 172.21.15.43:0/1690350717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac415b7d0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac9000640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 0x7f5ac40b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5abbe00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c460 tx=0x7f5aac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac017070 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac005ce0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac415bab0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac007290 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.411+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac4156550 con 0x7f5ac40b5b10 2024-04-26T20:26:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.413+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac415bfd0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.414+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5aac01a070 con 0x7f5ac40b5b10 2024-04-26T20:26:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.415+0000 7f5ac0c00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.415+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5aac08b1a0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.415+0000 7f5ac3e00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.416+0000 7f5ac3e00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4034000 tx=0x7f5ab402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.419+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ac415bfd0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.627+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5ac40506c0 con 0x7f5ac40b5b10 2024-04-26T20:26:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.628+0000 7f5ac0c00640 1 -- 172.21.15.43:0/1690350717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f5aac059320 con 0x7f5ac40b5b10 2024-04-26T20:26:35.630 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 msgr2=0x7f5aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 0x7f5aa006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4034000 tx=0x7f5ab402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 msgr2=0x7f5ac415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c460 tx=0x7f5aac00c920 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 shutdown_connections 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5aa006c8f0 0x7f5aa006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ac4155cd0 0x7f5ac40b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac40b5b10 0x7f5ac415b150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 --2- 172.21.15.43:0/1690350717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ac4072b40 0x7f5ac415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 >> 172.21.15.43:0/1690350717 conn(0x7f5ac4068090 msgr2=0x7f5ac4153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 shutdown_connections 2024-04-26T20:26:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.631+0000 7f5ac9a00640 1 -- 172.21.15.43:0/1690350717 wait complete. 2024-04-26T20:26:35.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:35.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.866+0000 7ff8ef200640 1 Processor -- start 2024-04-26T20:26:35.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.866+0000 7ff8ef200640 1 -- start start 2024-04-26T20:26:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.867+0000 7ff8ef200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8f00d5750 0x7ff8f00d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.867+0000 7ff8ef200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f00d0480 con 0x7ff8f00d1620 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.868+0000 7ff8ef200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f00d0720 con 0x7ff8f00b5b80 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.868+0000 7ff8ef200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f00d5be0 con 0x7ff8f00d5750 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.868+0000 7ff8ee800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8f00d5750 0x7ff8f00d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.868+0000 7ff8ede00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff8f00d1620 0x7ff8f00d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40662/0 (socket says 172.21.15.43:40662) 2024-04-26T20:26:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.868+0000 7ff8ede00640 1 -- 172.21.15.43:0/1990938452 learned_addr learned my addr 172.21.15.43:0/1990938452 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.869+0000 7ff8ee800640 1 -- 172.21.15.43:0/1990938452 >> v1:172.21.15.164:6789/0 conn(0x7ff8f00b5b80 legacy=0x7ff8f00d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.869+0000 7ff8ee800640 1 -- 172.21.15.43:0/1990938452 >> v1:172.21.15.43:6789/0 conn(0x7ff8f00d1620 legacy=0x7ff8f00d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.869+0000 7ff8ee800640 1 -- 172.21.15.43:0/1990938452 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00d0a00 con 0x7ff8f00d5750 2024-04-26T20:26:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.869+0000 7ff8eca00640 1 -- 172.21.15.43:0/1990938452 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975133170 0 0) 0x7ff8f00d0480 con 0x7ff8f00d1620 2024-04-26T20:26:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.870+0000 7ff8ee800640 1 --2- 172.21.15.43:0/1990938452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00d5750 0x7ff8f00d2f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097a0 tx=0x7ff8e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49486ebc3f9b881c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.870+0000 7ff8eca00640 1 -- 172.21.15.43:0/1990938452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8037070 con 0x7ff8f00d5750 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.870+0000 7ff8eca00640 1 -- 172.21.15.43:0/1990938452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e8032440 con 0x7ff8f00d5750 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.870+0000 7ff8eca00640 1 -- 172.21.15.43:0/1990938452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8032810 con 0x7ff8f00d5750 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 -- 172.21.15.43:0/1990938452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00d5750 msgr2=0x7ff8f00d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1990938452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00d5750 0x7ff8f00d2f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80097a0 tx=0x7ff8e802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 -- 172.21.15.43:0/1990938452 shutdown_connections 2024-04-26T20:26:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1990938452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00d5750 0x7ff8f00d2f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 -- 172.21.15.43:0/1990938452 >> 172.21.15.43:0/1990938452 conn(0x7ff8f00d4c30 msgr2=0x7ff8f00d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 -- 172.21.15.43:0/1990938452 shutdown_connections 2024-04-26T20:26:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.871+0000 7ff8ef200640 1 -- 172.21.15.43:0/1990938452 wait complete. 2024-04-26T20:26:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.873+0000 7ff8ef200640 1 Processor -- start 2024-04-26T20:26:35.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.873+0000 7ff8ef200640 1 -- start start 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8f00d1620 0x7ff8f01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 0x7ff8f00b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00d3450 con 0x7ff8f00b5b80 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00d0ce0 con 0x7ff8f00d5750 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ef200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f00d35d0 con 0x7ff8f00d1620 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36614/0 (socket says 172.21.15.43:36614) 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.874+0000 7ff8ede00640 1 -- 172.21.15.43:0/1574314366 learned_addr learned my addr 172.21.15.43:0/1574314366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ee800640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 0x7ff8f00b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ed400640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8f00d1620 0x7ff8f01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 -- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8f00d1620 msgr2=0x7ff8f01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8f00d1620 0x7ff8f01587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 -- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 msgr2=0x7ff8f00b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 0x7ff8f00b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 -- 172.21.15.43:0/1574314366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00c8590 con 0x7ff8f00b5b80 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ee800640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 0x7ff8f00b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ede00640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c8c0 tx=0x7ff8e000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0018070 con 0x7ff8f00b5b80 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e000a640 con 0x7ff8f00b5b80 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000aa70 con 0x7ff8f00b5b80 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.875+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f00b3090 con 0x7ff8f00b5b80 2024-04-26T20:26:35.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.876+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f015beb0 con 0x7ff8f00b5b80 2024-04-26T20:26:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.877+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f0069350 con 0x7ff8f00b5b80 2024-04-26T20:26:35.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.878+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8e001b070 con 0x7ff8f00b5b80 2024-04-26T20:26:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.879+0000 7ff8e6000640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.879+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff8e008b590 con 0x7ff8f00b5b80 2024-04-26T20:26:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.879+0000 7ff8ed400640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:35.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.880+0000 7ff8ed400640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8f015bde0 tx=0x7ff8d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:35.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:35.883+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8e00595e0 con 0x7ff8f00b5b80 2024-04-26T20:26:36.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.136+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f00008d0 con 0x7ff8f00b5b80 2024-04-26T20:26:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.137+0000 7ff8e6000640 1 -- 172.21.15.43:0/1574314366 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7ff8e0058ee0 con 0x7ff8f00b5b80 2024-04-26T20:26:36.139 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:36.139 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 msgr2=0x7ff8cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8f015bde0 tx=0x7ff8d8009040 comp rx=0 tx=0).stop 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 msgr2=0x7ff8f015dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c8c0 tx=0x7ff8e000cd80 comp rx=0 tx=0).stop 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 shutdown_connections 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8f00d1620 0x7ff8f01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8f00d5750 0x7ff8f00b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 --2- 172.21.15.43:0/1574314366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f00b5b80 0x7ff8f015dd70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 >> 172.21.15.43:0/1574314366 conn(0x7ff8f00d4c30 msgr2=0x7ff8f00d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.140+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 shutdown_connections 2024-04-26T20:26:36.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.141+0000 7ff8ef200640 1 -- 172.21.15.43:0/1574314366 wait complete. 2024-04-26T20:26:36.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:36.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.362+0000 7f8a3cc00640 1 Processor -- start 2024-04-26T20:26:36.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.362+0000 7f8a3cc00640 1 -- start start 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a3cc00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a38072620 0x7f8a380cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a3cc00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a38072a80 con 0x7f8a380ce2b0 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a3cc00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a380cd560 con 0x7f8a380cd180 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a3cc00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8a380ce6e0 con 0x7f8a38072620 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a2ea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a38072620 0x7f8a380cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a37e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a380ce2b0 0x7f8a3806dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40676/0 (socket says 172.21.15.43:40676) 2024-04-26T20:26:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a37e00640 1 -- 172.21.15.43:0/2291174725 learned_addr learned my addr 172.21.15.43:0/2291174725 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a2ea00640 1 -- 172.21.15.43:0/2291174725 >> v1:172.21.15.164:6789/0 conn(0x7f8a380cd180 legacy=0x7f8a3815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a2ea00640 1 -- 172.21.15.43:0/2291174725 >> v1:172.21.15.43:6789/0 conn(0x7f8a380ce2b0 legacy=0x7f8a3806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.364+0000 7f8a2ea00640 1 -- 172.21.15.43:0/2291174725 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a38072d00 con 0x7f8a38072620 2024-04-26T20:26:36.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.366+0000 7f8a2ea00640 1 --2- 172.21.15.43:0/2291174725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a38072620 0x7f8a380cdd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009d90 tx=0x7f8a2002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996e3fc3c4822395 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.366+0000 7f8a36a00640 1 -- 172.21.15.43:0/2291174725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20007740 con 0x7f8a38072620 2024-04-26T20:26:36.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.366+0000 7f8a36a00640 1 -- 172.21.15.43:0/2291174725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a200078e0 con 0x7f8a38072620 2024-04-26T20:26:36.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.366+0000 7f8a36a00640 1 -- 172.21.15.43:0/2291174725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20007cb0 con 0x7f8a38072620 2024-04-26T20:26:36.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.366+0000 7f8a3cc00640 1 -- 172.21.15.43:0/2291174725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a38072620 msgr2=0x7f8a380cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/2291174725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a38072620 0x7f8a380cdd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009d90 tx=0x7f8a2002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 -- 172.21.15.43:0/2291174725 shutdown_connections 2024-04-26T20:26:36.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/2291174725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a38072620 0x7f8a380cdd70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 -- 172.21.15.43:0/2291174725 >> 172.21.15.43:0/2291174725 conn(0x7f8a38068090 msgr2=0x7f8a38068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 -- 172.21.15.43:0/2291174725 shutdown_connections 2024-04-26T20:26:36.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.367+0000 7f8a3cc00640 1 -- 172.21.15.43:0/2291174725 wait complete. 2024-04-26T20:26:36.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.369+0000 7f8a3cc00640 1 Processor -- start 2024-04-26T20:26:36.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.369+0000 7f8a3cc00640 1 -- start start 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 0x7f8a3813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a380ce2b0 0x7f8a381f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380735a0 con 0x7f8a380cd180 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380732c0 con 0x7f8a38072620 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a3cc00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8a38073da0 con 0x7f8a380ce2b0 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36628/0 (socket says 172.21.15.43:36628) 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 -- 172.21.15.43:0/4160676273 learned_addr learned my addr 172.21.15.43:0/4160676273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:36.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a37400640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 0x7f8a3813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 -- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a380ce2b0 msgr2=0x7f8a381f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a37e00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a380ce2b0 0x7f8a381f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a380ce2b0 0x7f8a381f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 -- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 msgr2=0x7f8a3813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 0x7f8a3813ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a2ea00640 1 -- 172.21.15.43:0/4160676273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3815bf00 con 0x7f8a380cd180 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.370+0000 7f8a37400640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 0x7f8a3813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a2ea00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009d90 tx=0x7f8a20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20040070 con 0x7f8a380cd180 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20031d50 con 0x7f8a380cd180 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20045660 con 0x7f8a380cd180 2024-04-26T20:26:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3813f060 con 0x7f8a380cd180 2024-04-26T20:26:36.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.371+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3813f620 con 0x7f8a380cd180 2024-04-26T20:26:36.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.372+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a38069350 con 0x7f8a380cd180 2024-04-26T20:26:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.373+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a20039040 con 0x7f8a380cd180 2024-04-26T20:26:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.374+0000 7f8a2fe00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 0x7f8a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.374+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8a200ad8e0 con 0x7f8a380cd180 2024-04-26T20:26:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.374+0000 7f8a37400640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 0x7f8a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.375+0000 7f8a37400640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 0x7f8a1406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a28003eb0 tx=0x7f8a28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.378+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a2007bae0 con 0x7f8a380cd180 2024-04-26T20:26:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.582+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a380720e0 con 0x7f8a380cd180 2024-04-26T20:26:36.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.584+0000 7f8a2fe00640 1 -- 172.21.15.43:0/4160676273 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f8a2007b3e0 con 0x7f8a380cd180 2024-04-26T20:26:36.586 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:36.586 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.586+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 msgr2=0x7f8a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 0x7f8a1406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a28003eb0 tx=0x7f8a28009040 comp rx=0 tx=0).stop 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 msgr2=0x7f8a3815b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009d90 tx=0x7f8a20004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 shutdown_connections 2024-04-26T20:26:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8a1406c8f0 0x7f8a1406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a380ce2b0 0x7f8a381f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a380cd180 0x7f8a3815b9c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 --2- 172.21.15.43:0/4160676273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a38072620 0x7f8a3813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 >> 172.21.15.43:0/4160676273 conn(0x7f8a38068090 msgr2=0x7f8a38068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 shutdown_connections 2024-04-26T20:26:36.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.587+0000 7f8a3cc00640 1 -- 172.21.15.43:0/4160676273 wait complete. 2024-04-26T20:26:36.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:36.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.815+0000 7f4c1e600640 1 Processor -- start 2024-04-26T20:26:36.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.815+0000 7f4c1e600640 1 -- start start 2024-04-26T20:26:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1e600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c1815a7e0 0x7f4c180b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1e600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4c180b5410 0x7f4c180b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1e600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c180b3900 con 0x7f4c1815d2b0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1e600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4c1815abc0 con 0x7f4c1815a7e0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1e600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4c180b3b70 con 0x7f4c180b5410 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1d200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c1815a7e0 0x7f4c180b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1dc00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c1815d2b0 0x7f4c1815d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40690/0 (socket says 172.21.15.43:40690) 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.817+0000 7f4c1dc00640 1 -- 172.21.15.43:0/3919434877 learned_addr learned my addr 172.21.15.43:0/3919434877 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c0fe00640 1 --2- 172.21.15.43:0/3919434877 >> v2:172.21.15.164:3300/0 conn(0x7f4c180b5410 0x7f4c180b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c17e00640 1 -- 172.21.15.43:0/3919434877 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2717301820 0 0) 0x7f4c180b3900 con 0x7f4c1815d2b0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c17e00640 1 -- 172.21.15.43:0/3919434877 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bfc003520 con 0x7f4c1815d2b0 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c1d200640 1 -- 172.21.15.43:0/3919434877 >> v2:172.21.15.164:3300/0 conn(0x7f4c180b5410 msgr2=0x7f4c180b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c1d200640 1 --2- 172.21.15.43:0/3919434877 >> v2:172.21.15.164:3300/0 conn(0x7f4c180b5410 0x7f4c180b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c1d200640 1 -- 172.21.15.43:0/3919434877 >> v1:172.21.15.43:6789/0 conn(0x7f4c1815d2b0 legacy=0x7f4c1815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.818+0000 7f4c1d200640 1 -- 172.21.15.43:0/3919434877 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c180b3e50 con 0x7f4c1815a7e0 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.819+0000 7f4c1d200640 1 --2- 172.21.15.43:0/3919434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1815a7e0 0x7f4c180b4650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c180d15f0 tx=0x7f4c0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a49e6ee6aa5e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.819+0000 7f4c17e00640 1 -- 172.21.15.43:0/3919434877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08037070 con 0x7f4c1815a7e0 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.819+0000 7f4c17e00640 1 -- 172.21.15.43:0/3919434877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08032440 con 0x7f4c1815a7e0 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.819+0000 7f4c17e00640 1 -- 172.21.15.43:0/3919434877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08032810 con 0x7f4c1815a7e0 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 -- 172.21.15.43:0/3919434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1815a7e0 msgr2=0x7f4c180b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 --2- 172.21.15.43:0/3919434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1815a7e0 0x7f4c180b4650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c180d15f0 tx=0x7f4c0802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 -- 172.21.15.43:0/3919434877 shutdown_connections 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 --2- 172.21.15.43:0/3919434877 >> v2:172.21.15.164:3300/0 conn(0x7f4c180b5410 0x7f4c180b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 --2- 172.21.15.43:0/3919434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1815a7e0 0x7f4c180b4650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 -- 172.21.15.43:0/3919434877 >> 172.21.15.43:0/3919434877 conn(0x7f4c18068090 msgr2=0x7f4c180684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 -- 172.21.15.43:0/3919434877 shutdown_connections 2024-04-26T20:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.820+0000 7f4c1e600640 1 -- 172.21.15.43:0/3919434877 wait complete. 2024-04-26T20:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.822+0000 7f4c1e600640 1 Processor -- start 2024-04-26T20:26:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.822+0000 7f4c1e600640 1 -- start start 2024-04-26T20:26:36.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1815a7e0 0x7f4c180aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c1815d2b0 0x7f4c180aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4c180b4b90 con 0x7f4c180b5410 2024-04-26T20:26:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1815e010 con 0x7f4c1815a7e0 2024-04-26T20:26:36.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.823+0000 7f4c1e600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1815e2e0 con 0x7f4c1815d2b0 2024-04-26T20:26:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36640/0 (socket says 172.21.15.43:36640) 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 -- 172.21.15.43:0/4275149379 learned_addr learned my addr 172.21.15.43:0/4275149379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1dc00640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c1815d2b0 0x7f4c180aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c0fe00640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1815a7e0 0x7f4c180aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 -- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c1815d2b0 msgr2=0x7f4c180aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c1815d2b0 0x7f4c180aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 -- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1815a7e0 msgr2=0x7f4c180aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1815a7e0 0x7f4c180aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.824+0000 7f4c1d200640 1 -- 172.21.15.43:0/4275149379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c180acfd0 con 0x7f4c180b5410 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c1d200640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000c00 tx=0x7f4c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08041020 con 0x7f4c180b5410 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0803d700 con 0x7f4c180b5410 2024-04-26T20:26:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08037040 con 0x7f4c180b5410 2024-04-26T20:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c181e6420 con 0x7f4c180b5410 2024-04-26T20:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.825+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c180ab5c0 con 0x7f4c180b5410 2024-04-26T20:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.826+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c180506c0 con 0x7f4c180b5410 2024-04-26T20:26:36.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.827+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c08044020 con 0x7f4c180b5410 2024-04-26T20:26:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.828+0000 7f4c15600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.828+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4c080ac690 con 0x7f4c180b5410 2024-04-26T20:26:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.828+0000 7f4c0fe00640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.829+0000 7f4c0fe00640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c180cc980 tx=0x7f4c00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:36.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:36.833+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c0804b030 con 0x7f4c180b5410 2024-04-26T20:26:37.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.030+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c1814d6d0 con 0x7f4c180b5410 2024-04-26T20:26:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.031+0000 7f4c15600640 1 -- 172.21.15.43:0/4275149379 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f4c180acfd0 con 0x7f4c180b5410 2024-04-26T20:26:37.033 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.034+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 msgr2=0x7f4bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.034+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c180cc980 tx=0x7f4c00008040 comp rx=0 tx=0).stop 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.034+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 msgr2=0x7f4c1814d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.034+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c08000c00 tx=0x7f4c080046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 shutdown_connections 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c1815d2b0 0x7f4c180aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c1815a7e0 0x7f4c180aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 --2- 172.21.15.43:0/4275149379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c180b5410 0x7f4c1814d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 >> 172.21.15.43:0/4275149379 conn(0x7f4c18068090 msgr2=0x7f4c180b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 shutdown_connections 2024-04-26T20:26:37.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.035+0000 7f4c1e600640 1 -- 172.21.15.43:0/4275149379 wait complete. 2024-04-26T20:26:37.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.251+0000 7ff855600640 1 Processor -- start 2024-04-26T20:26:37.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.251+0000 7ff855600640 1 -- start start 2024-04-26T20:26:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.252+0000 7ff855600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff85015f1b0 0x7ff850075730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.252+0000 7ff855600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8500ce240 0x7ff8500aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.252+0000 7ff855600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff850075dc0 con 0x7ff8500b3810 2024-04-26T20:26:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.252+0000 7ff855600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff85015f590 con 0x7ff85015f1b0 2024-04-26T20:26:37.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.252+0000 7ff855600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500ae9a0 con 0x7ff8500ce240 2024-04-26T20:26:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.253+0000 7ff854c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8500ce240 0x7ff8500aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.253+0000 7ff84fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff85015f1b0 0x7ff850075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.253+0000 7ff84fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff85015f1b0 0x7ff850075730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:60738/0 (socket says 172.21.15.43:60738) 2024-04-26T20:26:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.253+0000 7ff84fe00640 1 -- 172.21.15.43:0/1244807 learned_addr learned my addr 172.21.15.43:0/1244807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:37.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.254+0000 7ff84fe00640 1 -- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8500ce240 msgr2=0x7ff8500aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.254+0000 7ff84fe00640 1 --2- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8500ce240 0x7ff8500aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.254+0000 7ff84fe00640 1 -- 172.21.15.43:0/1244807 >> v1:172.21.15.164:6789/0 conn(0x7ff8500b3810 legacy=0x7ff8500cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.254+0000 7ff84fe00640 1 -- 172.21.15.43:0/1244807 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500ae070 con 0x7ff85015f1b0 2024-04-26T20:26:37.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.255+0000 7ff84fe00640 1 --2- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85015f1b0 0x7ff850075730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff840009870 tx=0x7ff84002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5223f35823eece2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff84ea00640 1 -- 172.21.15.43:0/1244807 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840037070 con 0x7ff85015f1b0 2024-04-26T20:26:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff84ea00640 1 -- 172.21.15.43:0/1244807 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff840032440 con 0x7ff85015f1b0 2024-04-26T20:26:37.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff84ea00640 1 -- 172.21.15.43:0/1244807 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840032810 con 0x7ff85015f1b0 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 -- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85015f1b0 msgr2=0x7ff850075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 --2- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85015f1b0 0x7ff850075730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff840009870 tx=0x7ff84002a250 comp rx=0 tx=0).stop 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 -- 172.21.15.43:0/1244807 shutdown_connections 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 --2- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8500ce240 0x7ff8500aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 --2- 172.21.15.43:0/1244807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff85015f1b0 0x7ff850075730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 -- 172.21.15.43:0/1244807 >> 172.21.15.43:0/1244807 conn(0x7ff8500b2cf0 msgr2=0x7ff8500b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 -- 172.21.15.43:0/1244807 shutdown_connections 2024-04-26T20:26:37.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.256+0000 7ff855600640 1 -- 172.21.15.43:0/1244807 wait complete. 2024-04-26T20:26:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.258+0000 7ff855600640 1 Processor -- start 2024-04-26T20:26:37.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.258+0000 7ff855600640 1 -- start start 2024-04-26T20:26:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500b3810 0x7ff850157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff85015f1b0 0x7ff8501ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500cb770 con 0x7ff8500ce240 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500ae630 con 0x7ff8500b3810 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff855600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500d0e70 con 0x7ff85015f1b0 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.259+0000 7ff84f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36654/0 (socket says 172.21.15.43:36654) 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 -- 172.21.15.43:0/83108285 learned_addr learned my addr 172.21.15.43:0/83108285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff854c00640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff85015f1b0 0x7ff8501ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84fe00640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500b3810 0x7ff850157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 -- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff85015f1b0 msgr2=0x7ff8501ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff85015f1b0 0x7ff8501ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 -- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500b3810 msgr2=0x7ff850157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500b3810 0x7ff850157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 -- 172.21.15.43:0/83108285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff850157bc0 con 0x7ff8500ce240 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.260+0000 7ff84f400640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c440 tx=0x7ff83c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.262+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0052d0 con 0x7ff8500ce240 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.262+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c005470 con 0x7ff8500ce240 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.262+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8500c66a0 con 0x7ff8500ce240 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.262+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c007260 con 0x7ff8500ce240 2024-04-26T20:26:37.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.262+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8500c6c60 con 0x7ff8500ce240 2024-04-26T20:26:37.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.263+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8500506c0 con 0x7ff8500ce240 2024-04-26T20:26:37.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.264+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff83c0074c0 con 0x7ff8500ce240 2024-04-26T20:26:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.265+0000 7ff847e00640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 0x7ff83006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.265+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff83c08b3b0 con 0x7ff8500ce240 2024-04-26T20:26:37.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.265+0000 7ff84fe00640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 0x7ff83006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.266+0000 7ff84fe00640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 0x7ff83006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff840034000 tx=0x7ff84002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.269+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff850157bc0 con 0x7ff8500ce240 2024-04-26T20:26:37.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.472+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff850073510 con 0x7ff8500ce240 2024-04-26T20:26:37.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.473+0000 7ff847e00640 1 -- 172.21.15.43:0/83108285 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff83c059400 con 0x7ff8500ce240 2024-04-26T20:26:37.474 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.475+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 msgr2=0x7ff83006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.475+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 0x7ff83006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff840034000 tx=0x7ff84002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.475+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 msgr2=0x7ff8501ea410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.475+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c440 tx=0x7ff83c00c900 comp rx=0 tx=0).stop 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 shutdown_connections 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff83006c8f0 0x7ff83006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff85015f1b0 0x7ff8501ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8500ce240 0x7ff8501ea410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 --2- 172.21.15.43:0/83108285 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8500b3810 0x7ff850157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 >> 172.21.15.43:0/83108285 conn(0x7ff8500b2cf0 msgr2=0x7ff85006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 shutdown_connections 2024-04-26T20:26:37.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.476+0000 7ff855600640 1 -- 172.21.15.43:0/83108285 wait complete. 2024-04-26T20:26:37.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:37.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.708+0000 7f47fb600640 1 Processor -- start 2024-04-26T20:26:37.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.709+0000 7f47fb600640 1 -- start start 2024-04-26T20:26:37.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.710+0000 7f47fb600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 0x7f47f40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.710+0000 7f47fb600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f47f415d5e0 0x7f47f40cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.710+0000 7f47fb600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f4146160 con 0x7f47f4145d80 2024-04-26T20:26:37.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.710+0000 7f47fb600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f47f406dfb0 con 0x7f47f415d5e0 2024-04-26T20:26:37.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.710+0000 7f47fb600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f47f4074180 con 0x7f47f4072a90 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.711+0000 7f47f8c00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f47f4145d80 0x7f47f4072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:48158/0 (socket says 172.21.15.43:48158) 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.711+0000 7f47f8c00640 1 -- 172.21.15.43:0/3935855296 learned_addr learned my addr 172.21.15.43:0/3935855296 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.711+0000 7f47f9600640 1 --2- 172.21.15.43:0/3935855296 >> v2:172.21.15.164:3300/0 conn(0x7f47f415d5e0 0x7f47f40cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.711+0000 7f47f3e00640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 0x7f47f40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f3400640 1 -- 172.21.15.43:0/3935855296 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660587621 0 0) 0x7f47f4146160 con 0x7f47f4145d80 2024-04-26T20:26:37.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f3400640 1 -- 172.21.15.43:0/3935855296 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47d8003520 con 0x7f47f4145d80 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f9600640 1 -- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 msgr2=0x7f47f40cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f9600640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 0x7f47f40cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f9600640 1 -- 172.21.15.43:0/3935855296 >> v1:172.21.15.164:6789/0 conn(0x7f47f4145d80 legacy=0x7f47f4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f9600640 1 -- 172.21.15.43:0/3935855296 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40d0710 con 0x7f47f415d5e0 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.712+0000 7f47f3e00640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 0x7f47f40cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:37.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.713+0000 7f47f9600640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f415d5e0 0x7f47f40cf9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d90 tx=0x7f47ec02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c0f79cde7cef42a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47f3400640 1 -- 172.21.15.43:0/3935855296 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec007740 con 0x7f47f415d5e0 2024-04-26T20:26:37.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47f3400640 1 -- 172.21.15.43:0/3935855296 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0078e0 con 0x7f47f415d5e0 2024-04-26T20:26:37.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47f3400640 1 -- 172.21.15.43:0/3935855296 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec007cb0 con 0x7f47f415d5e0 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 -- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f415d5e0 msgr2=0x7f47f40cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f415d5e0 0x7f47f40cf9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009d90 tx=0x7f47ec02a660 comp rx=0 tx=0).stop 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 -- 172.21.15.43:0/3935855296 shutdown_connections 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f415d5e0 0x7f47f40cf9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 --2- 172.21.15.43:0/3935855296 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4072a90 0x7f47f40cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 -- 172.21.15.43:0/3935855296 >> 172.21.15.43:0/3935855296 conn(0x7f47f40b2900 msgr2=0x7f47f40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 -- 172.21.15.43:0/3935855296 shutdown_connections 2024-04-26T20:26:37.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.714+0000 7f47fb600640 1 -- 172.21.15.43:0/3935855296 wait complete. 2024-04-26T20:26:37.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.716+0000 7f47fb600640 1 Processor -- start 2024-04-26T20:26:37.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.716+0000 7f47fb600640 1 -- start start 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4072a90 0x7f47f413f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4145d80 0x7f47f413c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40d09f0 con 0x7f47f415d5e0 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47f406e530 con 0x7f47f4072a90 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47fb600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f47f41468c0 con 0x7f47f4145d80 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47f9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47f9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36666/0 (socket says 172.21.15.43:36666) 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.717+0000 7f47f9600640 1 -- 172.21.15.43:0/3726849067 learned_addr learned my addr 172.21.15.43:0/3726849067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f8c00640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4072a90 0x7f47f413f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f3e00640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4145d80 0x7f47f413c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 -- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4145d80 msgr2=0x7f47f413c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4145d80 0x7f47f413c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 -- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4072a90 msgr2=0x7f47f413f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4072a90 0x7f47f413f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 -- 172.21.15.43:0/3726849067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f413fb00 con 0x7f47f415d5e0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f9600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009eb0 tx=0x7f47ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.718+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec03f070 con 0x7f47f415d5e0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.719+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec031bc0 con 0x7f47f415d5e0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.719+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec009450 con 0x7f47f415d5e0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.719+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f4140ca0 con 0x7f47f415d5e0 2024-04-26T20:26:37.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.719+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f413edc0 con 0x7f47f415d5e0 2024-04-26T20:26:37.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.720+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f413e8d0 con 0x7f47f415d5e0 2024-04-26T20:26:37.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.721+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47ec03b410 con 0x7f47f415d5e0 2024-04-26T20:26:37.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.723+0000 7f47f0c00640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 0x7f47d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:37.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.723+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f47ec0ad620 con 0x7f47f415d5e0 2024-04-26T20:26:37.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.723+0000 7f47f8c00640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 0x7f47d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:37.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.724+0000 7f47f8c00640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 0x7f47d406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47e40011e0 tx=0x7f47e4007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:37.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.728+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47ec07b670 con 0x7f47f415d5e0 2024-04-26T20:26:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.930+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47f4069350 con 0x7f47f415d5e0 2024-04-26T20:26:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.930+0000 7f47f0c00640 1 -- 172.21.15.43:0/3726849067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f47ec07af70 con 0x7f47f415d5e0 2024-04-26T20:26:37.932 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 msgr2=0x7f47d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 0x7f47d406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47e40011e0 tx=0x7f47e4007d30 comp rx=0 tx=0).stop 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 msgr2=0x7f47f413c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009eb0 tx=0x7f47ec004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 shutdown_connections 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47d406c8f0 0x7f47d406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47f4145d80 0x7f47f413c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47f415d5e0 0x7f47f413c850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 --2- 172.21.15.43:0/3726849067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47f4072a90 0x7f47f413f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.933+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 >> 172.21.15.43:0/3726849067 conn(0x7f47f40b2900 msgr2=0x7f47f4144f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.934+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 shutdown_connections 2024-04-26T20:26:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:37.934+0000 7f47fb600640 1 -- 172.21.15.43:0/3726849067 wait complete. 2024-04-26T20:26:37.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:38.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.165+0000 7fc67f200640 1 Processor -- start 2024-04-26T20:26:38.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.166+0000 7fc67f200640 1 -- start start 2024-04-26T20:26:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.167+0000 7fc67f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc678163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.167+0000 7fc67f200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc67806f380 0x7fc6780b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.167+0000 7fc67f200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc678073660 con 0x7fc6780723f0 2024-04-26T20:26:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.167+0000 7fc67f200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc67806df40 con 0x7fc67806f380 2024-04-26T20:26:38.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.167+0000 7fc67f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc67806e190 con 0x7fc67814d200 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.168+0000 7fc67d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc67806f380 0x7fc6780b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.168+0000 7fc67d200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc67806f380 0x7fc6780b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:60764/0 (socket says 172.21.15.43:60764) 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.168+0000 7fc67d200640 1 -- 172.21.15.43:0/2867081714 learned_addr learned my addr 172.21.15.43:0/2867081714 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.168+0000 7fc677e00640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc678163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.169+0000 7fc67d200640 1 -- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 msgr2=0x7fc678163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.169+0000 7fc67d200640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc678163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.169+0000 7fc67d200640 1 -- 172.21.15.43:0/2867081714 >> v1:172.21.15.43:6789/0 conn(0x7fc6780723f0 legacy=0x7fc67806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.169+0000 7fc67d200640 1 -- 172.21.15.43:0/2867081714 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67806e470 con 0x7fc67806f380 2024-04-26T20:26:38.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.170+0000 7fc67d200640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc6780b3c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc670009e50 tx=0x7fc67002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9373bdf7d9ad347b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc677400640 1 -- 172.21.15.43:0/2867081714 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670036070 con 0x7fc67806f380 2024-04-26T20:26:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc677400640 1 -- 172.21.15.43:0/2867081714 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc670031600 con 0x7fc67806f380 2024-04-26T20:26:38.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc677400640 1 -- 172.21.15.43:0/2867081714 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6700319d0 con 0x7fc67806f380 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 -- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 msgr2=0x7fc6780b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc6780b3c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc670009e50 tx=0x7fc67002a710 comp rx=0 tx=0).stop 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 -- 172.21.15.43:0/2867081714 shutdown_connections 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc6780b3c00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 --2- 172.21.15.43:0/2867081714 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc678163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 -- 172.21.15.43:0/2867081714 >> 172.21.15.43:0/2867081714 conn(0x7fc6780b2900 msgr2=0x7fc6780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 -- 172.21.15.43:0/2867081714 shutdown_connections 2024-04-26T20:26:38.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.171+0000 7fc67f200640 1 -- 172.21.15.43:0/2867081714 wait complete. 2024-04-26T20:26:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.173+0000 7fc67f200640 1 Processor -- start 2024-04-26T20:26:38.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.173+0000 7fc67f200640 1 -- start start 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc678163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc67814abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc678073440 con 0x7fc6780723f0 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780732c0 con 0x7fc67806f380 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc67f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc67806f810 con 0x7fc67814d200 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.174+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36672/0 (socket says 172.21.15.43:36672) 2024-04-26T20:26:38.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 -- 172.21.15.43:0/4294198809 learned_addr learned my addr 172.21.15.43:0/4294198809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc677e00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc678163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc67d200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc67814abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 -- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 msgr2=0x7fc67814abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc67814abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 -- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 msgr2=0x7fc678163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc678163140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 -- 172.21.15.43:0/4294198809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6780ab7a0 con 0x7fc6780723f0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc677e00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc678163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.175+0000 7fc66fe00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c430 tx=0x7fc66000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.176+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6600052d0 con 0x7fc6780723f0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.176+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc660005490 con 0x7fc6780723f0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.176+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660007260 con 0x7fc6780723f0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.176+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6780aba80 con 0x7fc6780723f0 2024-04-26T20:26:38.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.176+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6780afb00 con 0x7fc6780723f0 2024-04-26T20:26:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.177+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc678069350 con 0x7fc6780723f0 2024-04-26T20:26:38.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.178+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc660007400 con 0x7fc6780723f0 2024-04-26T20:26:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.179+0000 7fc674c00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 0x7fc65406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.179+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc66008b540 con 0x7fc6780723f0 2024-04-26T20:26:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.179+0000 7fc677e00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 0x7fc65406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.180+0000 7fc677e00640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 0x7fc65406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc668009ac0 tx=0x7fc668008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.183+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc660059510 con 0x7fc6780723f0 2024-04-26T20:26:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.386+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc6780b5970 con 0x7fc6780723f0 2024-04-26T20:26:38.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.387+0000 7fc674c00640 1 -- 172.21.15.43:0/4294198809 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fc660058e10 con 0x7fc6780723f0 2024-04-26T20:26:38.388 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 msgr2=0x7fc65406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 0x7fc65406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc668009ac0 tx=0x7fc668008040 comp rx=0 tx=0).stop 2024-04-26T20:26:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 msgr2=0x7fc67814a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c430 tx=0x7fc66000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 shutdown_connections 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc65406c8f0 0x7fc65406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc67814d200 0x7fc67814abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6780723f0 0x7fc67814a6a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 --2- 172.21.15.43:0/4294198809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc67806f380 0x7fc678163140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 >> 172.21.15.43:0/4294198809 conn(0x7fc6780b2900 msgr2=0x7fc678163f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 shutdown_connections 2024-04-26T20:26:38.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.390+0000 7fc67f200640 1 -- 172.21.15.43:0/4294198809 wait complete. 2024-04-26T20:26:38.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:38.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.618+0000 7f24f6a00640 1 Processor -- start 2024-04-26T20:26:38.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.618+0000 7f24f6a00640 1 -- start start 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.620+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f0145590 0x7f24f00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.620+0000 7f24f6a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f24f006f730 0x7f24f0071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f6a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f00cf060 con 0x7f24f00b3490 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f6a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f24f00d0100 con 0x7f24f006f730 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f6a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00cf340 con 0x7f24f0145590 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f4a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f24f006f730 0x7f24f0071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24ef400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f24f00b3490 0x7f24f006f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40704/0 (socket says 172.21.15.43:40704) 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24ef400640 1 -- 172.21.15.43:0/3639045611 learned_addr learned my addr 172.21.15.43:0/3639045611 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24efe00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f0145590 0x7f24f00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f4a00640 1 -- 172.21.15.43:0/3639045611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f0145590 msgr2=0x7f24f00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f4a00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f0145590 0x7f24f00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f4a00640 1 -- 172.21.15.43:0/3639045611 >> v1:172.21.15.43:6789/0 conn(0x7f24f00b3490 legacy=0x7f24f006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24f4a00640 1 -- 172.21.15.43:0/3639045611 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f00cf620 con 0x7f24f006f730 2024-04-26T20:26:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.621+0000 7f24eea00640 1 -- 172.21.15.43:0/3639045611 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360666811 0 0) 0x7f24f00cf060 con 0x7f24f00b3490 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.622+0000 7f24f4a00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f006f730 0x7f24f0071eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097a0 tx=0x7f24e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd0d9c5b3cfa472 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24eea00640 1 -- 172.21.15.43:0/3639045611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8037070 con 0x7f24f006f730 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24eea00640 1 -- 172.21.15.43:0/3639045611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8032440 con 0x7f24f006f730 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24eea00640 1 -- 172.21.15.43:0/3639045611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8032810 con 0x7f24f006f730 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 -- 172.21.15.43:0/3639045611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f006f730 msgr2=0x7f24f0071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f006f730 0x7f24f0071eb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097a0 tx=0x7f24e802a180 comp rx=0 tx=0).stop 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 -- 172.21.15.43:0/3639045611 shutdown_connections 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f006f730 0x7f24f0071eb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 --2- 172.21.15.43:0/3639045611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f0145590 0x7f24f00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.623+0000 7f24f6a00640 1 -- 172.21.15.43:0/3639045611 >> 172.21.15.43:0/3639045611 conn(0x7f24f00b2900 msgr2=0x7f24f00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.624+0000 7f24f6a00640 1 -- 172.21.15.43:0/3639045611 shutdown_connections 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.624+0000 7f24f6a00640 1 -- 172.21.15.43:0/3639045611 wait complete. 2024-04-26T20:26:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.625+0000 7f24f6a00640 1 Processor -- start 2024-04-26T20:26:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.625+0000 7f24f6a00640 1 -- start start 2024-04-26T20:26:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24f006f730 0x7f24f013f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f00b3490 0x7f24f013a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0145970 con 0x7f24f0145590 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24f00cfbe0 con 0x7f24f006f730 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.626+0000 7f24f6a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f24f00cecc0 con 0x7f24f00b3490 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36680/0 (socket says 172.21.15.43:36680) 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 -- 172.21.15.43:0/2250555826 learned_addr learned my addr 172.21.15.43:0/2250555826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24efe00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24f006f730 0x7f24f013f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24ef400640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f00b3490 0x7f24f013a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 -- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f00b3490 msgr2=0x7f24f013a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f00b3490 0x7f24f013a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 -- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24f006f730 msgr2=0x7f24f013f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24f006f730 0x7f24f013f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.627+0000 7f24f4a00640 1 -- 172.21.15.43:0/2250555826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f013abd0 con 0x7f24f0145590 2024-04-26T20:26:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24f4a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097a0 tx=0x7f24e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8037040 con 0x7f24f0145590 2024-04-26T20:26:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8042070 con 0x7f24f0145590 2024-04-26T20:26:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f013aeb0 con 0x7f24f0145590 2024-04-26T20:26:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8030b30 con 0x7f24f0145590 2024-04-26T20:26:38.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.628+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f013b750 con 0x7f24f0145590 2024-04-26T20:26:38.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.629+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f013b080 con 0x7f24f0145590 2024-04-26T20:26:38.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.630+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e8030cd0 con 0x7f24f0145590 2024-04-26T20:26:38.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.631+0000 7f24e7e00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:38.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.631+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24e80ac980 con 0x7f24f0145590 2024-04-26T20:26:38.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.631+0000 7f24efe00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:38.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.632+0000 7f24efe00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 0x7f24d006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24f013e9f0 tx=0x7f24e00039b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:38.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.635+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24e8032e20 con 0x7f24f0145590 2024-04-26T20:26:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.880+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f013b980 con 0x7f24f0145590 2024-04-26T20:26:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.882+0000 7f24e7e00640 1 -- 172.21.15.43:0/2250555826 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f24e8045360 con 0x7f24f0145590 2024-04-26T20:26:38.883 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:38.883 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 msgr2=0x7f24d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 0x7f24d006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24f013e9f0 tx=0x7f24e00039b0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 msgr2=0x7f24f0139710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097a0 tx=0x7f24e80046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 shutdown_connections 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f24d006c8f0 0x7f24d006edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24f00b3490 0x7f24f013a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24f0145590 0x7f24f0139710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 --2- 172.21.15.43:0/2250555826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24f006f730 0x7f24f013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:38.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 >> 172.21.15.43:0/2250555826 conn(0x7f24f00b2900 msgr2=0x7f24f0144830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:38.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.884+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 shutdown_connections 2024-04-26T20:26:38.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:38.885+0000 7f24f6a00640 1 -- 172.21.15.43:0/2250555826 wait complete. 2024-04-26T20:26:38.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:39.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.118+0000 7f3bf0c00640 1 Processor -- start 2024-04-26T20:26:39.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.118+0000 7f3bf0c00640 1 -- start start 2024-04-26T20:26:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bf0c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3bec072b70 0x7f3bec0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bf0c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bec146ba0 con 0x7f3bec0d1060 2024-04-26T20:26:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bf0c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bec0731c0 con 0x7f3bec1467c0 2024-04-26T20:26:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bf0c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3bec073dd0 con 0x7f3bec072b70 2024-04-26T20:26:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bea800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3bec0d1060 0x7f3bec06dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40716/0 (socket says 172.21.15.43:40716) 2024-04-26T20:26:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.120+0000 7f3bea800640 1 -- 172.21.15.43:0/2908627087 learned_addr learned my addr 172.21.15.43:0/2908627087 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be9400640 1 --2- 172.21.15.43:0/2908627087 >> v2:172.21.15.164:3300/0 conn(0x7f3bec072b70 0x7f3bec0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245560706 0 0) 0x7f3bec146ba0 con 0x7f3bec0d1060 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bd0003520 con 0x7f3bec0d1060 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2590500994 0 0) 0x7f3bd0003520 con 0x7f3bec0d1060 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 >> v2:172.21.15.164:3300/0 conn(0x7f3bec072b70 msgr2=0x7f3bec0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 --2- 172.21.15.43:0/2908627087 >> v2:172.21.15.164:3300/0 conn(0x7f3bec072b70 0x7f3bec0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 >> v1:172.21.15.164:6789/0 conn(0x7f3bec1467c0 legacy=0x7f3bec0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec0734a0 con 0x7f3bec0d1060 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f3be40019f0 con 0x7f3bec0d1060 2024-04-26T20:26:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.121+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 >> v1:172.21.15.43:6789/0 conn(0x7f3bec0d1060 legacy=0x7f3bec06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec073dd0 con 0x7f3bd0004800 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd000c060 con 0x7f3bd00051c0 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd000c260 con 0x7f3bd00039b0 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3bea800640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9e00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 -- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd00039b0 msgr2=0x7f3bd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 -- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bd00051c0 msgr2=0x7f3bd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3be9400640 1 -- 172.21.15.43:0/2908627087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd0003520 con 0x7f3bd0004800 2024-04-26T20:26:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3bea800640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.122+0000 7f3bea800640 1 -- 172.21.15.43:0/2908627087 reap_dead start 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.123+0000 7f3be9400640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009ac0 tx=0x7f3bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6acce7f7ba25c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8036070 con 0x7f3bd0004800 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd8031400 con 0x7f3bd0004800 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3be8a00640 1 -- 172.21.15.43:0/2908627087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8009690 con 0x7f3bd0004800 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3bf0c00640 1 -- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 msgr2=0x7f3bd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009ac0 tx=0x7f3bd0005830 comp rx=0 tx=0).stop 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3bf0c00640 1 -- 172.21.15.43:0/2908627087 shutdown_connections 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/2908627087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.124+0000 7f3bf0c00640 1 -- 172.21.15.43:0/2908627087 >> 172.21.15.43:0/2908627087 conn(0x7f3bec0b2900 msgr2=0x7f3bec0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.125+0000 7f3bf0c00640 1 -- 172.21.15.43:0/2908627087 shutdown_connections 2024-04-26T20:26:39.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.125+0000 7f3bf0c00640 1 -- 172.21.15.43:0/2908627087 wait complete. 2024-04-26T20:26:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.126+0000 7f3bf0c00640 1 Processor -- start 2024-04-26T20:26:39.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.127+0000 7f3bf0c00640 1 -- start start 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd0004800 0x7f3bec138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 0x7f3bec0cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec0d0b20 con 0x7f3bec1e7110 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec073a60 con 0x7f3bec0cf660 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bf0c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bec0d0ca0 con 0x7f3bd0004800 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36694/0 (socket says 172.21.15.43:36694) 2024-04-26T20:26:39.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 -- 172.21.15.43:0/1946001144 learned_addr learned my addr 172.21.15.43:0/1946001144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3be9e00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd0004800 0x7f3bec138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3be9400640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 0x7f3bec0cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 -- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd0004800 msgr2=0x7f3bec138440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd0004800 0x7f3bec138440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 -- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 msgr2=0x7f3bec0cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 0x7f3bec0cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3bea800640 1 -- 172.21.15.43:0/1946001144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec138980 con 0x7f3bec1e7110 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.128+0000 7f3be9400640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 0x7f3bec0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:39.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3bea800640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3be40012e0 tx=0x7f3be400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004d60 con 0x7f3bec1e7110 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be400bb80 con 0x7f3bec1e7110 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4007110 con 0x7f3bec1e7110 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec1e7e80 con 0x7f3bec1e7110 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.129+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1e8440 con 0x7f3bec1e7110 2024-04-26T20:26:39.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.131+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bec069350 con 0x7f3bec1e7110 2024-04-26T20:26:39.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.131+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3be401b020 con 0x7f3bec1e7110 2024-04-26T20:26:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.132+0000 7f3be2000640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 0x7f3bcc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.133+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3be408bfe0 con 0x7f3bec1e7110 2024-04-26T20:26:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.133+0000 7f3be9e00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 0x7f3bcc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.133+0000 7f3be9e00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 0x7f3bcc06ee00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001710 tx=0x7f3bdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.137+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3be405a180 con 0x7f3bec1e7110 2024-04-26T20:26:39.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.349+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bec0afd10 con 0x7f3bec1e7110 2024-04-26T20:26:39.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.351+0000 7f3be2000640 1 -- 172.21.15.43:0/1946001144 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18490 (secure 0 0 0) 0x7f3be4059a80 con 0x7f3bec1e7110 2024-04-26T20:26:39.352 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:39.353 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:02.669052+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.353+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 msgr2=0x7f3bcc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.353+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 0x7f3bcc06ee00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001710 tx=0x7f3bdc009040 comp rx=0 tx=0).stop 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.353+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 msgr2=0x7f3bec1e7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.353+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3be40012e0 tx=0x7f3be400b4a0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 shutdown_connections 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3bcc06c940 0x7f3bcc06ee00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bec1e7110 0x7f3bec1e7590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bec0cf660 0x7f3bec0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 --2- 172.21.15.43:0/1946001144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bd0004800 0x7f3bec138440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 >> 172.21.15.43:0/1946001144 conn(0x7f3bec0b2900 msgr2=0x7f3bec0b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:39.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 shutdown_connections 2024-04-26T20:26:39.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.354+0000 7f3bf0c00640 1 -- 172.21.15.43:0/1946001144 wait complete. 2024-04-26T20:26:39.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:26:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.588+0000 7f9b1fe00640 1 Processor -- start 2024-04-26T20:26:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.589+0000 7f9b1fe00640 1 -- start start 2024-04-26T20:26:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9b200d1620 0x7f9b200d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200d1a00 0x7f9b200b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200b50b0 0x7f9b200b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9b200d0ce0 con 0x7f9b200d1a00 2024-04-26T20:26:39.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9b200d0480 con 0x7f9b200d1620 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.590+0000 7f9b1fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9b200d0720 con 0x7f9b200b50b0 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200d1a00 0x7f9b200b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b200d1a00 0x7f9b200b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36704/0 (socket says 172.21.15.43:36704) 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 -- 172.21.15.43:0/2083388485 learned_addr learned my addr 172.21.15.43:0/2083388485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1ea00640 1 --2- 172.21.15.43:0/2083388485 >> v2:172.21.15.164:3300/0 conn(0x7f9b200d1620 0x7f9b200d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1f400640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200b50b0 0x7f9b200b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 -- 172.21.15.43:0/2083388485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200b50b0 msgr2=0x7f9b200b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:39.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200b50b0 0x7f9b200b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 -- 172.21.15.43:0/2083388485 >> v2:172.21.15.164:3300/0 conn(0x7f9b200d1620 msgr2=0x7f9b200d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 --2- 172.21.15.43:0/2083388485 >> v2:172.21.15.164:3300/0 conn(0x7f9b200d1620 0x7f9b200d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.591+0000 7f9b1e000640 1 -- 172.21.15.43:0/2083388485 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b200d0a00 con 0x7f9b200d1a00 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.593+0000 7f9b1e000640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200d1a00 0x7f9b200b4b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b2008d330 tx=0x7f9b0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cef2cfb412f826 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.593+0000 7f9b1d600640 1 -- 172.21.15.43:0/2083388485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c037070 con 0x7f9b200d1a00 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.593+0000 7f9b1d600640 1 -- 172.21.15.43:0/2083388485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c032440 con 0x7f9b200d1a00 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.593+0000 7f9b1d600640 1 -- 172.21.15.43:0/2083388485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c032810 con 0x7f9b200d1a00 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 -- 172.21.15.43:0/2083388485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200d1a00 msgr2=0x7f9b200b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200d1a00 0x7f9b200b4b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b2008d330 tx=0x7f9b0c02a140 comp rx=0 tx=0).stop 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 -- 172.21.15.43:0/2083388485 shutdown_connections 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200b50b0 0x7f9b200b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/2083388485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200d1a00 0x7f9b200b4b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/2083388485 >> v2:172.21.15.164:3300/0 conn(0x7f9b200d1620 0x7f9b200d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 -- 172.21.15.43:0/2083388485 >> 172.21.15.43:0/2083388485 conn(0x7f9b200d4c30 msgr2=0x7f9b200d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 -- 172.21.15.43:0/2083388485 shutdown_connections 2024-04-26T20:26:39.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.594+0000 7f9b1fe00640 1 -- 172.21.15.43:0/2083388485 wait complete. 2024-04-26T20:26:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.596+0000 7f9b1fe00640 1 Processor -- start 2024-04-26T20:26:39.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.596+0000 7f9b1fe00640 1 -- start start 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200d1620 0x7f9b200b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b200d1a00 0x7f9b201591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9b200b5a80 con 0x7f9b200b50b0 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b200d1e60 con 0x7f9b200d1a00 2024-04-26T20:26:39.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9b200b5c00 con 0x7f9b200d1620 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36708/0 (socket says 172.21.15.43:36708) 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1ea00640 1 -- 172.21.15.43:0/4149400586 learned_addr learned my addr 172.21.15.43:0/4149400586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.597+0000 7f9b1e000640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200d1620 0x7f9b200b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1f400640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b200d1a00 0x7f9b201591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 -- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200d1620 msgr2=0x7f9b200b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200d1620 0x7f9b200b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 -- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b200d1a00 msgr2=0x7f9b201591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b200d1a00 0x7f9b201591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 -- 172.21.15.43:0/4149400586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b200c9660 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b1ea00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b10007e40 tx=0x7f9b1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.598+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10014620 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.599+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b10019070 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.599+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b10014a00 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.599+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b200c9940 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.599+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b200c9f00 con 0x7f9b200b50b0 2024-04-26T20:26:39.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.600+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b20069350 con 0x7f9b200b50b0 2024-04-26T20:26:39.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.601+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b10014ba0 con 0x7f9b200b50b0 2024-04-26T20:26:39.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.602+0000 7f9b16a00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 0x7f9b0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:39.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.602+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9b1008c790 con 0x7f9b200b50b0 2024-04-26T20:26:39.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.602+0000 7f9b1e000640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 0x7f9b0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:39.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.603+0000 7f9b1e000640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 0x7f9b0006ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c034000 tx=0x7f9b0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.606+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b1005a7e0 con 0x7f9b200b50b0 2024-04-26T20:26:39.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.850+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b201596e0 con 0x7f9b200b50b0 2024-04-26T20:26:39.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.851+0000 7f9b16a00640 1 -- 172.21.15.43:0/4149400586 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9b1005a0e0 con 0x7f9b200b50b0 2024-04-26T20:26:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:39.853 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:26:39.854 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:26:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.854+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 msgr2=0x7f9b0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.854+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 0x7f9b0006ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c034000 tx=0x7f9b0c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.854+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 msgr2=0x7f9b200c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.854+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b10007e40 tx=0x7f9b1000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 shutdown_connections 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9b0006c920 0x7f9b0006ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b200d1620 0x7f9b200b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b200d1a00 0x7f9b201591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 --2- 172.21.15.43:0/4149400586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b200b50b0 0x7f9b200c8fa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 >> 172.21.15.43:0/4149400586 conn(0x7f9b200d4c30 msgr2=0x7f9b2006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 shutdown_connections 2024-04-26T20:26:39.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:39.855+0000 7f9b1fe00640 1 -- 172.21.15.43:0/4149400586 wait complete. 2024-04-26T20:26:40.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:25:57.479955+0000', 'modified': '2024-04-26T20:26:24.666310+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.164:6835/2023737548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6833', 'nonce': 2023737548}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2023737548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4273, 'qdb_cluster': [4273]} 2024-04-26T20:26:40.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:26:40.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:26:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.498+0000 7f923fe00640 1 Processor -- start 2024-04-26T20:26:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.499+0000 7f923fe00640 1 -- start start 2024-04-26T20:26:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.500+0000 7f923fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f924015bb90 0x7f92400737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.500+0000 7f923fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.500+0000 7f923fe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9240072db0 con 0x7f92400af710 2024-04-26T20:26:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923fe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f92400cec10 con 0x7f924015bb90 2024-04-26T20:26:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f92400725a0 con 0x7f92400afb70 2024-04-26T20:26:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923ea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f924015bb90 0x7f92400737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923e000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f92400af710 0x7f924015f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40724/0 (socket says 172.21.15.43:40724) 2024-04-26T20:26:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923e000640 1 -- 172.21.15.43:0/2147377954 learned_addr learned my addr 172.21.15.43:0/2147377954 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923f400640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923ea00640 1 -- 172.21.15.43:0/2147377954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 msgr2=0x7f924015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923ea00640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.501+0000 7f923ea00640 1 -- 172.21.15.43:0/2147377954 >> v1:172.21.15.43:6789/0 conn(0x7f92400af710 legacy=0x7f924015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.502+0000 7f923ea00640 1 -- 172.21.15.43:0/2147377954 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92400cee40 con 0x7f924015bb90 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.502+0000 7f923d600640 1 -- 172.21.15.43:0/2147377954 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305499710 0 0) 0x7f9240072db0 con 0x7f92400af710 2024-04-26T20:26:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.502+0000 7f923f400640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.503+0000 7f923ea00640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f92400737a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92300097a0 tx=0x7f923002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296f69697d2622cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.503+0000 7f923d600640 1 -- 172.21.15.43:0/2147377954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230037070 con 0x7f924015bb90 2024-04-26T20:26:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.503+0000 7f923d600640 1 -- 172.21.15.43:0/2147377954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9230032440 con 0x7f924015bb90 2024-04-26T20:26:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.503+0000 7f923d600640 1 -- 172.21.15.43:0/2147377954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230032810 con 0x7f924015bb90 2024-04-26T20:26:40.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 -- 172.21.15.43:0/2147377954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 msgr2=0x7f92400737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f92400737a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92300097a0 tx=0x7f923002a180 comp rx=0 tx=0).stop 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 -- 172.21.15.43:0/2147377954 shutdown_connections 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 --2- 172.21.15.43:0/2147377954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f92400737a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 -- 172.21.15.43:0/2147377954 >> 172.21.15.43:0/2147377954 conn(0x7f9240068090 msgr2=0x7f9240068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 -- 172.21.15.43:0/2147377954 shutdown_connections 2024-04-26T20:26:40.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.504+0000 7f923fe00640 1 -- 172.21.15.43:0/2147377954 wait complete. 2024-04-26T20:26:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.506+0000 7f923fe00640 1 Processor -- start 2024-04-26T20:26:40.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.506+0000 7f923fe00640 1 -- start start 2024-04-26T20:26:40.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 0x7f92401f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924014f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9240072ad0 con 0x7f924015bb90 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92400d0610 con 0x7f92400af710 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f92400cf0e0 con 0x7f92400afb70 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36724/0 (socket says 172.21.15.43:36724) 2024-04-26T20:26:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.507+0000 7f923f400640 1 -- 172.21.15.43:0/1014903808 learned_addr learned my addr 172.21.15.43:0/1014903808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:40.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 -- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 msgr2=0x7f924014f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923e000640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923ea00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 0x7f92401f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924014f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 -- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 msgr2=0x7f92401f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 0x7f92401f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 -- 172.21.15.43:0/1014903808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9240157da0 con 0x7f924015bb90 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923e000640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923ea00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 0x7f92401f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923f400640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f923800cf20 tx=0x7f923800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f9237400640 1 -- 172.21.15.43:0/1014903808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923801a070 con 0x7f924015bb90 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f9237400640 1 -- 172.21.15.43:0/1014903808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923800d840 con 0x7f924015bb90 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f9237400640 1 -- 172.21.15.43:0/1014903808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923800dc30 con 0x7f924015bb90 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.508+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9240158080 con 0x7f924015bb90 2024-04-26T20:26:40.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.509+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9240158640 con 0x7f924015bb90 2024-04-26T20:26:40.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.511+0000 7f9237400640 1 -- 172.21.15.43:0/1014903808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f923801d070 con 0x7f924015bb90 2024-04-26T20:26:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.512+0000 7f9237400640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.512+0000 7f9237400640 1 -- 172.21.15.43:0/1014903808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f923808c760 con 0x7f924015bb90 2024-04-26T20:26:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.512+0000 7f923ea00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.513+0000 7f9248dc5740 1 Processor -- start 2024-04-26T20:26:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.513+0000 7f9248dc5740 1 -- start start 2024-04-26T20:26:40.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.513+0000 7f923ea00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 0x7f922006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9230034000 tx=0x7f923002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 0x55f1aa56c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa4d1600 con 0x55f1aa5955a0 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa46e100 con 0x55f1aa56f950 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55f1aa4cd5c0 con 0x55f1aa5b4900 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36732/0 (socket says 172.21.15.43:36732) 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923ea00640 1 -- 172.21.15.43:0/892951153 learned_addr learned my addr 172.21.15.43:0/892951153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923f400640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 0x55f1aa56c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923e000640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.514+0000 7f923ea00640 1 -- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 msgr2=0x55f1aa56c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923ea00640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 0x55f1aa56c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923ea00640 1 -- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 msgr2=0x55f1aa5b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923ea00640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923ea00640 1 -- 172.21.15.43:0/892951153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1aa1e32d0 con 0x55f1aa5955a0 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923f400640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 0x55f1aa56c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923e000640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f923ea00640 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9230035040 tx=0x7f923002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a4d58a24ed19d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f922b400640 1 -- 172.21.15.43:0/892951153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923004d070 con 0x55f1aa5955a0 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f922b400640 1 -- 172.21.15.43:0/892951153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9230048470 con 0x55f1aa5955a0 2024-04-26T20:26:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.515+0000 7f922b400640 1 -- 172.21.15.43:0/892951153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230037040 con 0x55f1aa5955a0 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 -- 172.21.15.43:0/892951153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 msgr2=0x55f1aa595980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9230035040 tx=0x7f923002c000 comp rx=0 tx=0).stop 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 -- 172.21.15.43:0/892951153 shutdown_connections 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b4900 0x55f1aa56c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 --2- 172.21.15.43:0/892951153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa595980 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 -- 172.21.15.43:0/892951153 >> 172.21.15.43:0/892951153 conn(0x55f1aa501d10 msgr2=0x55f1aa5b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 -- 172.21.15.43:0/892951153 shutdown_connections 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.516+0000 7f9248dc5740 1 -- 172.21.15.43:0/892951153 wait complete. 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.517+0000 7f9248dc5740 1 -- start start 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.517+0000 7f9248dc5740 1 Processor -- start 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 0x55f1aa5b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa569db0 con 0x55f1aa5955a0 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa458f70 con 0x55f1aa56f950 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55f1aa56e8e0 con 0x55f1aa5b5250 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f9248dc5740 10 client.0 _ll_register_callbacks cb 0x55f1aa4c7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f923e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f923e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36738/0 (socket says 172.21.15.43:36738) 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.518+0000 7f923e000640 1 -- 172.21.15.43:0/859370237 learned_addr learned my addr 172.21.15.43:0/859370237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923f400640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 0x55f1aa5b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923ea00640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 -- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 msgr2=0x55f1aa5b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 0x55f1aa5b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 -- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 msgr2=0x55f1aa5ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 -- 172.21.15.43:0/859370237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1aa2ad6b0 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923f400640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 0x55f1aa5b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f923e000640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f922c00e1c0 tx=0x7f922c00e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1027e33794cd66c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.519+0000 7f921fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c019060 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f922c0074e0 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9230048470 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f922c0047e0 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 10 client.4845 fetch_fsmap learned FSMap version 9 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55f1aa58ff80 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9230048470 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f922c004980 con 0x55f1aa5955a0 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 10 client.4845 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 10 client.4845 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:26:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.520+0000 7f9248dc5740 20 client.4845 populate_metadata read hostname 'smithi043' 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.521+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa57b520 0x55f1aa57b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.521+0000 7f9248dc5740 4 client.4845 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.521+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 --> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1aa4df210 con 0x55f1aa57b520 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.521+0000 7f923ea00640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa57b520 0x55f1aa57b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.522+0000 7f923ea00640 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa57b520 0x55f1aa57b900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8248ca1e687a17da server_cookie=f29e66c1b062836f in_seq=0 out_seq=0 2024-04-26T20:26:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.522+0000 7f921fe00640 10 client.4845 ms_handle_connect on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f921fe00640 1 -- 172.21.15.43:0/859370237 <== mds.0 v2:172.21.15.164:6833/2023737548 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1aa4df210 con 0x55f1aa57b520 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f921fe00640 10 client.4845 handle_command_reply: tid=0 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 client.4845 shutdown 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 msgr2=0x55f1aa57d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f922c00e1c0 tx=0x7f922c00e680 comp rx=0 tx=0).stop 2024-04-26T20:26:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 shutdown_connections 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa57b520 0x55f1aa57b900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5b5250 0x55f1aa5b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5955a0 0x55f1aa57d780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 --2- 172.21.15.43:0/859370237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa5ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f921fe00640 0 client.4845 ms_handle_reset on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 >> 172.21.15.43:0/859370237 conn(0x55f1aa501d10 msgr2=0x55f1aa56de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 shutdown_connections 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.524+0000 7f9248dc5740 1 -- 172.21.15.43:0/859370237 wait complete. 2024-04-26T20:26:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.525+0000 7f9248dc5740 20 client.4845 trim_cache size 0 max 16384 2024-04-26T20:26:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.545+0000 7f9248dc5740 1 Processor -- start 2024-04-26T20:26:40.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.545+0000 7f9248dc5740 1 -- start start 2024-04-26T20:26:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa56f950 0x55f1aa57d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 0x55f1aa590d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f92401fd0a0 con 0x55f1aa5b5250 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f924015df80 con 0x55f1aa5955a0 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9240155850 con 0x55f1aa56f950 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36740/0 (socket says 172.21.15.43:36740) 2024-04-26T20:26:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f923f400640 1 -- 172.21.15.43:0/504084331 learned_addr learned my addr 172.21.15.43:0/504084331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f923e000640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 0x55f1aa590d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.546+0000 7f923ea00640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa56f950 0x55f1aa57d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 -- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa56f950 msgr2=0x55f1aa57d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa56f950 0x55f1aa57d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 -- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 msgr2=0x55f1aa590d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 0x55f1aa590d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 -- 172.21.15.43:0/504084331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1aa563410 con 0x55f1aa5b5250 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923e000640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 0x55f1aa590d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f923f400640 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f923800cfd0 tx=0x7f923805a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85afce6d9bb20a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f922aa00640 1 -- 172.21.15.43:0/504084331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92380581e0 con 0x55f1aa5b5250 2024-04-26T20:26:40.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f922aa00640 1 -- 172.21.15.43:0/504084331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238058440 con 0x55f1aa5b5250 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.547+0000 7f922aa00640 1 -- 172.21.15.43:0/504084331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92380567b0 con 0x55f1aa5b5250 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 -- 172.21.15.43:0/504084331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 msgr2=0x55f1aa5d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f923800cfd0 tx=0x7f923805a280 comp rx=0 tx=0).stop 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 -- 172.21.15.43:0/504084331 shutdown_connections 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa5d13c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa5955a0 0x55f1aa590d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 --2- 172.21.15.43:0/504084331 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa56f950 0x55f1aa57d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 -- 172.21.15.43:0/504084331 >> 172.21.15.43:0/504084331 conn(0x55f1aa501d10 msgr2=0x55f1aa567ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 -- 172.21.15.43:0/504084331 shutdown_connections 2024-04-26T20:26:40.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.548+0000 7f9248dc5740 1 -- 172.21.15.43:0/504084331 wait complete. 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.549+0000 7f9248dc5740 1 -- start start 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.549+0000 7f9248dc5740 1 Processor -- start 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa62ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5955a0 0x55f1aa6320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa58eee0 con 0x55f1aa5b5250 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f1aa57bf60 con 0x55f1aa56f950 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55f1aa5967b0 con 0x55f1aa5955a0 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f9248dc5740 10 client.0 _ll_register_callbacks cb 0x7f922c007c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f923f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36756/0 (socket says 172.21.15.43:36756) 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.550+0000 7f923f400640 1 -- 172.21.15.43:0/1680770078 learned_addr learned my addr 172.21.15.43:0/1680770078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923e000640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5955a0 0x55f1aa6320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923ea00640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa62ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 -- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5955a0 msgr2=0x55f1aa6320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5955a0 0x55f1aa6320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 -- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 msgr2=0x55f1aa62ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa62ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 -- 172.21.15.43:0/1680770078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1aa569aa0 con 0x55f1aa5b5250 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923ea00640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa62ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:40.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f923f400640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f923800acd0 tx=0x7f9238007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46331d4872f7edc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f921fe00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238062a90 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f923806eb50 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.551+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9238058440 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9238080050 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 10 client.4851 fetch_fsmap learned FSMap version 9 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55f1aa58e010 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9238058440 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9238059590 con 0x55f1aa5b5250 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 10 client.4851 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 10 client.4851 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 20 client.4851 populate_metadata read hostname 'smithi043' 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa580090 0x55f1aa580470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 4 client.4851 mds_command: new command op to 4273 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:26:40.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.552+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 --> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] -- command(tid 0: {"prefix": "status"}) -- 0x55f1aa56a0e0 con 0x55f1aa580090 2024-04-26T20:26:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.553+0000 7f923ea00640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa580090 0x55f1aa580470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.553+0000 7f923ea00640 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa580090 0x55f1aa580470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5e3fe6c1f6cd2 server_cookie=24e6c5dbc1f52d19 in_seq=0 out_seq=0 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.553+0000 7f921fe00640 10 client.4851 ms_handle_connect on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.554+0000 7f921fe00640 1 -- 172.21.15.43:0/1680770078 <== mds.0 v2:172.21.15.164:6833/2023737548 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55f1aa56a0e0 con 0x55f1aa580090 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.554+0000 7f921fe00640 10 client.4851 handle_command_reply: tid=0 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.554+0000 7f9248dc5740 1 client.4851 shutdown 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.554+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 msgr2=0x55f1aa58d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.554+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f923800acd0 tx=0x7f9238007040 comp rx=0 tx=0).stop 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 shutdown_connections 2024-04-26T20:26:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:6833/2023737548,v1:172.21.15.164:6835/2023737548] conn(0x55f1aa580090 0x55f1aa580470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55f1aa5955a0 0x55f1aa6320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1aa5b5250 0x55f1aa58d670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 --2- 172.21.15.43:0/1680770078 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f1aa56f950 0x55f1aa62ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f921fe00640 0 client.4851 ms_handle_reset on v2:172.21.15.164:6833/2023737548 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 >> 172.21.15.43:0/1680770078 conn(0x55f1aa501d10 msgr2=0x55f1aa57a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 shutdown_connections 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 1 -- 172.21.15.43:0/1680770078 wait complete. 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.555+0000 7f9248dc5740 20 client.4851 trim_cache size 0 max 16384 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6a354559-7839-4522-9cd7-4f1548bb621f", 2024-04-26T20:26:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4273, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 42.589718961999999, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 9, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 24, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 50.711508684999998 2024-04-26T20:26:40.560 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 msgr2=0x7f922006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 0x7f922006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9230034000 tx=0x7f923002e000 comp rx=0 tx=0).stop 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 msgr2=0x7f9240158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f923800cf20 tx=0x7f923800a7d0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 shutdown_connections 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f922006c8f0 0x7f922006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92400afb70 0x7f924014f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f924015bb90 0x7f9240158b60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 --2- 172.21.15.43:0/1014903808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92400af710 0x7f92401f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:40.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 >> 172.21.15.43:0/1014903808 conn(0x7f9240068090 msgr2=0x7f924015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:40.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 shutdown_connections 2024-04-26T20:26:40.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:40.556+0000 7f923fe00640 1 -- 172.21.15.43:0/1014903808 wait complete. 2024-04-26T20:26:40.576 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:26:40.585 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:26:40.585 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:26:40.586 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:26:40.586 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T20:26:40.597 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T20:26:40.597 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T20:26:40.598 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:40.598 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:40.598 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:40.598 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-26T20:26:40.598 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:26:40.599 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:40.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:40.599 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-04-26T20:26:40.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:40.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-04-26T20:26:40.743 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-04-26T20:26:40.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:26:40.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:40.747 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:26:40.807 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T20:26:40.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:40.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:26:40.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:40.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-26T20:26:41.015 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T20:26:41.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:26:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.198+0000 7f6006c00640 1 Processor -- start 2024-04-26T20:26:41.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.198+0000 7f6006c00640 1 -- start start 2024-04-26T20:26:41.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.200+0000 7f6006c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.200+0000 7f6006c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60000d13e0 0x7f60000d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.200+0000 7f6006c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6000163b40 con 0x7f60000cfc50 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.200+0000 7f6006c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f60000b3df0 con 0x7f60000d13e0 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.200+0000 7f6006c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f60000b40d0 con 0x7f60000b3850 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.201+0000 7f6004e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.201+0000 7f6006200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60000d13e0 0x7f60000d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.201+0000 7f6004e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:44876/0 (socket says 172.21.15.43:44876) 2024-04-26T20:26:41.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.201+0000 7f6006200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60000d13e0 0x7f60000d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:60862/0 (socket says 172.21.15.43:60862) 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.201+0000 7f6004e00640 1 -- 172.21.15.43:0/496242298 learned_addr learned my addr 172.21.15.43:0/496242298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.202+0000 7f6006200640 1 -- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 msgr2=0x7f600006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.202+0000 7f6006200640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.202+0000 7f6006200640 1 -- 172.21.15.43:0/496242298 >> v1:172.21.15.164:6789/0 conn(0x7f60000cfc50 legacy=0x7f60000b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.202+0000 7f6006200640 1 -- 172.21.15.43:0/496242298 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60000d1090 con 0x7f60000d13e0 2024-04-26T20:26:41.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.202+0000 7f6004e00640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:41.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.203+0000 7f6006200640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000d13e0 0x7f60000d1870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009d90 tx=0x7f5ff802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f57e4b7697254ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:41.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.204+0000 7f5fffe00640 1 -- 172.21.15.43:0/496242298 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8031600 con 0x7f60000d13e0 2024-04-26T20:26:41.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.204+0000 7f5fffe00640 1 -- 172.21.15.43:0/496242298 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff80317a0 con 0x7f60000d13e0 2024-04-26T20:26:41.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.204+0000 7f5fffe00640 1 -- 172.21.15.43:0/496242298 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8031b70 con 0x7f60000d13e0 2024-04-26T20:26:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.204+0000 7f6006c00640 1 -- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000d13e0 msgr2=0x7f60000d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.204+0000 7f6006c00640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000d13e0 0x7f60000d1870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009d90 tx=0x7f5ff802a5d0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 -- 172.21.15.43:0/496242298 shutdown_connections 2024-04-26T20:26:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000d13e0 0x7f60000d1870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 --2- 172.21.15.43:0/496242298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000b3850 0x7f600006eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 -- 172.21.15.43:0/496242298 >> 172.21.15.43:0/496242298 conn(0x7f60000b4600 msgr2=0x7f60000b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 -- 172.21.15.43:0/496242298 shutdown_connections 2024-04-26T20:26:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.205+0000 7f6006c00640 1 -- 172.21.15.43:0/496242298 wait complete. 2024-04-26T20:26:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.207+0000 7f6006c00640 1 Processor -- start 2024-04-26T20:26:41.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.207+0000 7f6006c00640 1 -- start start 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000b3850 0x7f60000b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000cfc50 0x7f60000c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6000163140 con 0x7f60000d13e0 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6000162fc0 con 0x7f60000b3850 2024-04-26T20:26:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f60001632c0 con 0x7f60000cfc50 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36762/0 (socket says 172.21.15.43:36762) 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6006200640 1 -- 172.21.15.43:0/2004593871 learned_addr learned my addr 172.21.15.43:0/2004593871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6005800640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000b3850 0x7f60000b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.209+0000 7f6004e00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000cfc50 0x7f60000c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 -- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000cfc50 msgr2=0x7f60000c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000cfc50 0x7f60000c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 -- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000b3850 msgr2=0x7f60000b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000b3850 0x7f60000b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 -- 172.21.15.43:0/2004593871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60000c7580 con 0x7f60000d13e0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.210+0000 7f6006200640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802abc0 tx=0x7f5ff8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.211+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff803f070 con 0x7f60000d13e0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.211+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff8030c00 con 0x7f60000d13e0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.211+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8009450 con 0x7f60000d13e0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.211+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60000c7860 con 0x7f60000d13e0 2024-04-26T20:26:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.211+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60000b5990 con 0x7f60000d13e0 2024-04-26T20:26:41.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.212+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6000069a60 con 0x7f60000d13e0 2024-04-26T20:26:41.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.213+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ff803b410 con 0x7f60000d13e0 2024-04-26T20:26:41.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.214+0000 7f5ffd600640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 0x7f5fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:41.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.214+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5ff80acb30 con 0x7f60000d13e0 2024-04-26T20:26:41.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.214+0000 7f6005800640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 0x7f5fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.215+0000 7f6005800640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 0x7f5fe006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4001780 tx=0x7f5ff4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.219+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ff807a5c0 con 0x7f60000d13e0 2024-04-26T20:26:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.466+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f60000cc6c0 con 0x7f60000d13e0 2024-04-26T20:26:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.467+0000 7f5ffd600640 1 -- 172.21.15.43:0/2004593871 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5ff807e8c0 con 0x7f60000d13e0 2024-04-26T20:26:41.468 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 msgr2=0x7f5fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 0x7f5fe006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4001780 tx=0x7f5ff4009040 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 msgr2=0x7f60000c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802abc0 tx=0x7f5ff8004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 shutdown_connections 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5fe006c8f0 0x7f5fe006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60000cfc50 0x7f60000c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60000d13e0 0x7f60000c6f50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 --2- 172.21.15.43:0/2004593871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60000b3850 0x7f60000b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 >> 172.21.15.43:0/2004593871 conn(0x7f60000b4600 msgr2=0x7f60000cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 shutdown_connections 2024-04-26T20:26:41.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41.470+0000 7f6006c00640 1 -- 172.21.15.43:0/2004593871 wait complete. 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:26:41.491 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:26:41.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:41.492 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-26T20:26:41.540 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:41.540 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:41.540 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:41.540 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:41.540 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:41.541 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:41.542 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:41.542 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-26T20:26:41.542 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:26:41.543 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:41.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:41.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:41.801 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:26:41.869 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-26T20:26:41.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:41.873 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-26T20:26:41.923 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-26T20:26:41.923 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:26:41.924 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:26:41.924 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:26:41.924 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:26:41.925 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:42.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:42.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:42.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:42.148 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:26:42.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:42.195 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-26T20:26:42.243 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:42.243 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:42.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:42.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:26:42.243 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:42.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:42.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:42.425 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:26:42.425 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:42.425 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:42.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:26:42.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:26:42.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:26:42.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:26:42.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:26:42.426 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:42.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:42.687 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:42.687 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:42.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-26T20:26:42.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:26:42.688 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:42.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:42.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:42.881 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:26:42.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:42.882 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:42.921 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:26:42.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:42.922 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:43.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:43.087 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:43.130 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:43.130 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:43.130 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:43.130 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:43.131 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:43.132 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:43.133 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:43.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:43.133 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:43.220 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:26:43.220 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:26:43.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:43.342 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:43.380 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:43.380 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:43.380 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:43.380 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:43.380 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:43.381 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:43.382 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:43.383 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:43.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:43.384 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:43.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-26T20:26:43.438+0000 7f353ec1f480 -1 init, newargv = 0x55e533c6e460 newargc=15 2024-04-26T20:26:43.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[42309]: starting ceph client 2024-04-26T20:26:43.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[42309]: starting fuse 2024-04-26T20:26:43.466 INFO:teuthology.orchestra.run.smithi043.stdout:42 2024-04-26T20:26:43.467 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:43.467 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-26T20:26:43.468 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-26T20:26:43.648 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.42309.asok 2024-04-26T20:26:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:43.664 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42309.asok 2024-04-26T20:26:43.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:43.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42309.asok status 2024-04-26T20:26:43.876 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:26:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-26T20:26:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:26:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "42309", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4860, 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4860 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 464275214 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 464275214 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4860 192.168.0.1:0/464275214", 2024-04-26T20:26:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/464275214", 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 25, 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-26T20:26:43.879 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:26:43.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.099+0000 7f1428a00640 1 Processor -- start 2024-04-26T20:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.099+0000 7f1428a00640 1 -- start start 2024-04-26T20:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.101+0000 7f1428a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 0x7f142415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.101+0000 7f1428a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1424161170 con 0x7f1424160ae0 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.101+0000 7f1428a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1424161450 con 0x7f142413fc90 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.101+0000 7f1428a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1424160f50 con 0x7f14240d14b0 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.102+0000 7f141be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1424160ae0 0x7f14240d0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40740/0 (socket says 172.21.15.43:40740) 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.102+0000 7f141be00640 1 -- 172.21.15.43:0/2950474250 learned_addr learned my addr 172.21.15.43:0/2950474250 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.102+0000 7f1422800640 1 --2- 172.21.15.43:0/2950474250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 0x7f142415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.102+0000 7f1422800640 1 -- 172.21.15.43:0/2950474250 >> v1:172.21.15.164:6789/0 conn(0x7f142413fc90 legacy=0x7f1424142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.102+0000 7f1421400640 1 -- 172.21.15.43:0/2950474250 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512812293 0 0) 0x7f1424161450 con 0x7f142413fc90 2024-04-26T20:26:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.103+0000 7f1422800640 1 -- 172.21.15.43:0/2950474250 >> v1:172.21.15.43:6789/0 conn(0x7f1424160ae0 legacy=0x7f14240d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.103+0000 7f1422800640 1 -- 172.21.15.43:0/2950474250 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1424161730 con 0x7f14240d14b0 2024-04-26T20:26:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.104+0000 7f1422800640 1 --2- 172.21.15.43:0/2950474250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 0x7f142415df70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f141c009dc0 tx=0x7f141c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1efeb1dd4c0a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.105+0000 7f1421400640 1 -- 172.21.15.43:0/2950474250 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c007740 con 0x7f14240d14b0 2024-04-26T20:26:44.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.105+0000 7f1421400640 1 -- 172.21.15.43:0/2950474250 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141c0078e0 con 0x7f14240d14b0 2024-04-26T20:26:44.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.105+0000 7f1421400640 1 -- 172.21.15.43:0/2950474250 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c007cb0 con 0x7f14240d14b0 2024-04-26T20:26:44.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 -- 172.21.15.43:0/2950474250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 msgr2=0x7f142415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 --2- 172.21.15.43:0/2950474250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 0x7f142415df70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f141c009dc0 tx=0x7f141c02a5d0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 -- 172.21.15.43:0/2950474250 shutdown_connections 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 --2- 172.21.15.43:0/2950474250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14240d14b0 0x7f142415df70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 -- 172.21.15.43:0/2950474250 >> 172.21.15.43:0/2950474250 conn(0x7f14240682a0 msgr2=0x7f14240686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 -- 172.21.15.43:0/2950474250 shutdown_connections 2024-04-26T20:26:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.106+0000 7f1428a00640 1 -- 172.21.15.43:0/2950474250 wait complete. 2024-04-26T20:26:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.107+0000 7f1428a00640 1 Processor -- start 2024-04-26T20:26:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.108+0000 7f1428a00640 1 -- start start 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.108+0000 7f1428a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14240d14b0 0x7f1424157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1428a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f142413fc90 0x7f142415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1428a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1428a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14241433a0 con 0x7f1424160ae0 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1428a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1424143520 con 0x7f14240d14b0 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1428a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1424140070 con 0x7f142413fc90 2024-04-26T20:26:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36774/0 (socket says 172.21.15.43:36774) 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 -- 172.21.15.43:0/1596322944 learned_addr learned my addr 172.21.15.43:0/1596322944 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1421e00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14240d14b0 0x7f1424157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f141be00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f142413fc90 0x7f142415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 -- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f142413fc90 msgr2=0x7f142415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f142413fc90 0x7f142415e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 -- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14240d14b0 msgr2=0x7f1424157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.109+0000 7f1422800640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14240d14b0 0x7f1424157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f1422800640 1 -- 172.21.15.43:0/1596322944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14241578b0 con 0x7f1424160ae0 2024-04-26T20:26:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f1422800640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f141c009ee0 tx=0x7f141c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c040070 con 0x7f1424160ae0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141c031c10 con 0x7f1424160ae0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1424155610 con 0x7f1424160ae0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c003990 con 0x7f1424160ae0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.110+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1424155bd0 con 0x7f1424160ae0 2024-04-26T20:26:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.112+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1424069350 con 0x7f1424160ae0 2024-04-26T20:26:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.113+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f141c039040 con 0x7f1424160ae0 2024-04-26T20:26:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.113+0000 7f141a000640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 0x7f140006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.114+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6772+0+0 (secure 0 0 0) 0x7f141c0ad5d0 con 0x7f1424160ae0 2024-04-26T20:26:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.114+0000 7f1421e00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 0x7f140006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:44.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.114+0000 7f1421e00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 0x7f140006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1414007fd0 tx=0x7f1414009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.118+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f141c07b530 con 0x7f1424160ae0 2024-04-26T20:26:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.373+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1424156f20 con 0x7f1424160ae0 2024-04-26T20:26:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.373+0000 7f141a000640 1 -- 172.21.15.43:0/1596322944 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f141c07ae30 con 0x7f1424160ae0 2024-04-26T20:26:44.374 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 msgr2=0x7f140006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 0x7f140006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1414007fd0 tx=0x7f1414009040 comp rx=0 tx=0).stop 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 msgr2=0x7f142415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f141c009ee0 tx=0x7f141c004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 shutdown_connections 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f140006c8f0 0x7f140006edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f142413fc90 0x7f142415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1424160ae0 0x7f142415eb60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 --2- 172.21.15.43:0/1596322944 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14240d14b0 0x7f1424157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 >> 172.21.15.43:0/1596322944 conn(0x7f14240682a0 msgr2=0x7f1424068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 shutdown_connections 2024-04-26T20:26:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44.376+0000 7f1428a00640 1 -- 172.21.15.43:0/1596322944 wait complete. 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:26:44.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:26:44.393 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:26:44.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:44.393 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-26T20:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-04-26T20:26:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 22:c5:59:a2:ed:8d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::68c8:72ff:fee1:53b7/64 scope link 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 22:c5:59:a2:ed:8d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::20c5:59ff:fea2:ed8d/64 scope link tentative 2024-04-26T20:26:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:44.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:44.438 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:26:44.548 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-26T20:26:44.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:44.552 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-26T20:26:44.599 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-26T20:26:44.599 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:26:44.600 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:26:44.600 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:26:44.600 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:26:44.601 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:44.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:44.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:44.825 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:26:44.875 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:26:44.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:44.877 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-26T20:26:44.933 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T20:26:44.934 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:44.934 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:44.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:44.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T20:26:44.935 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:45.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:45.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:45.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 72:e1:48:2c:c4:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::70e1:48ff:fe2c:c414/64 scope link 2024-04-26T20:26:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:45.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:26:45.290 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:45.290 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 72:e1:48:2c:c4:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::70e1:48ff:fe2c:c414/64 scope link 2024-04-26T20:26:45.291 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:45.295 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T20:26:45.295 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:45.295 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:45.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T20:26:45.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:26:45.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T20:26:45.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T20:26:45.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T20:26:45.296 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:45.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:45.583 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:45.583 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:26:45.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2024-04-26T20:26:45.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:26:45.583 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:26:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:45.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:45.801 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-26T20:26:45.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:45.801 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:45.849 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:26:45.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:45.849 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:45.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:45.990 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:46.050 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:46.050 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:46.050 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:46.050 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:46.050 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:46.052 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:46.053 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:46.054 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:46.054 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:46.054 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:46.054 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:46.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:46.055 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:46.106 INFO:teuthology.orchestra.run.smithi043.stdout:42 2024-04-26T20:26:46.107 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-26T20:26:46.107 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-26T20:26:46.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:26:46.184 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:26:46.226 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:26:46.227 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:46.228 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:26:46.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:46.229 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:26:46.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:2024-04-26T20:26:46.284+0000 7f25ce835480 -1 init, newargv = 0x5568f92c0460 newargc=15 2024-04-26T20:26:46.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[42808]: starting ceph client 2024-04-26T20:26:46.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[42808]: starting fuse 2024-04-26T20:26:46.315 INFO:teuthology.orchestra.run.smithi043.stdout:42 2024-04-26T20:26:46.315 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-26T20:26:46.315 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-26T20:26:46.315 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-26T20:26:46.316 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-26T20:26:46.317 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-26T20:26:46.317 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:26:46.317 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:26:46.317 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.2")) 2024-04-26T20:26:46.317 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-26T20:26:46.506 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.2.42808.asok 2024-04-26T20:26:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:46.524 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.42808.asok 2024-04-26T20:26:46.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:46.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.42808.asok status 2024-04-26T20:26:46.755 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "2", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "42808", 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-26T20:26:46.756 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4875, 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4875 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 476915939 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 476915939 2024-04-26T20:26:46.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4875 192.168.0.2:0/476915939", 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.2:0/476915939", 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 26, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-26T20:26:46.758 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:26:46.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:26:46.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.986+0000 7f0393e00640 1 Processor -- start 2024-04-26T20:26:46.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.987+0000 7f0393e00640 1 -- start start 2024-04-26T20:26:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.988+0000 7f0393e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.988+0000 7f0393e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 0x7f0394162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.988+0000 7f0393e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03940cf860 con 0x7f03940b5ca0 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.988+0000 7f0393e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f03940d3090 con 0x7f03940d1d90 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.988+0000 7f0393e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f03940d5b50 con 0x7f03940d3480 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.989+0000 7f0392a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.989+0000 7f0392000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 0x7f0394162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.989+0000 7f0392a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:47936/0 (socket says 172.21.15.43:47936) 2024-04-26T20:26:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.989+0000 7f0392a00640 1 -- 172.21.15.43:0/2854823448 learned_addr learned my addr 172.21.15.43:0/2854823448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.990+0000 7f0392a00640 1 -- 172.21.15.43:0/2854823448 >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 msgr2=0x7f0394162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.990+0000 7f0392a00640 1 --2- 172.21.15.43:0/2854823448 >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 0x7f0394162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.990+0000 7f0392a00640 1 -- 172.21.15.43:0/2854823448 >> v1:172.21.15.164:6789/0 conn(0x7f03940b5ca0 legacy=0x7f03940cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.990+0000 7f0392a00640 1 -- 172.21.15.43:0/2854823448 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03940cfb00 con 0x7f03940d3480 2024-04-26T20:26:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.990+0000 7f0392000640 1 --2- 172.21.15.43:0/2854823448 >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 0x7f0394162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.991+0000 7f0392a00640 1 --2- 172.21.15.43:0/2854823448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0384009870 tx=0x7f038402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55944a3e1edc8e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.992+0000 7f0391600640 1 -- 172.21.15.43:0/2854823448 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384037070 con 0x7f03940d3480 2024-04-26T20:26:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.992+0000 7f0391600640 1 -- 172.21.15.43:0/2854823448 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0384032440 con 0x7f03940d3480 2024-04-26T20:26:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.992+0000 7f0391600640 1 -- 172.21.15.43:0/2854823448 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384032810 con 0x7f03940d3480 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.992+0000 7f0393e00640 1 -- 172.21.15.43:0/2854823448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 msgr2=0x7f03940c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.992+0000 7f0393e00640 1 --2- 172.21.15.43:0/2854823448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0384009870 tx=0x7f038402a250 comp rx=0 tx=0).stop 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 -- 172.21.15.43:0/2854823448 shutdown_connections 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 --2- 172.21.15.43:0/2854823448 >> v2:172.21.15.164:3300/0 conn(0x7f03940d1d90 0x7f0394162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 --2- 172.21.15.43:0/2854823448 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940c87a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 -- 172.21.15.43:0/2854823448 >> 172.21.15.43:0/2854823448 conn(0x7f03940d4c50 msgr2=0x7f03940d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 -- 172.21.15.43:0/2854823448 shutdown_connections 2024-04-26T20:26:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.993+0000 7f0393e00640 1 -- 172.21.15.43:0/2854823448 wait complete. 2024-04-26T20:26:46.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.995+0000 7f0393e00640 1 Processor -- start 2024-04-26T20:26:46.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.995+0000 7f0393e00640 1 -- start start 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03940d1d90 0x7f03940cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d5770 con 0x7f03940b5ca0 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f03940d2db0 con 0x7f03940d1d90 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.996+0000 7f0393e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f03940d58f0 con 0x7f03940d3480 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37114/0 (socket says 172.21.15.43:37114) 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 -- 172.21.15.43:0/2452348322 learned_addr learned my addr 172.21.15.43:0/2452348322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392000640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03940d1d90 0x7f03940cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0393400640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 -- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 msgr2=0x7f03940a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:46.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:46.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 -- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03940d1d90 msgr2=0x7f03940cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:46.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03940d1d90 0x7f03940cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:46.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.997+0000 7f0392a00640 1 -- 172.21.15.43:0/2452348322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03940cc300 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f0392a00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f03840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384037040 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0384042070 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03940cc5e0 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384030880 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.998+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03940ccba0 con 0x7f03940b5ca0 2024-04-26T20:26:47.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:46.999+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0394069350 con 0x7f03940b5ca0 2024-04-26T20:26:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.001+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0384030a20 con 0x7f03940b5ca0 2024-04-26T20:26:47.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.002+0000 7f038aa00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 0x7f037006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:47.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.002+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f03840ac860 con 0x7f03940b5ca0 2024-04-26T20:26:47.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.002+0000 7f0392000640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 0x7f037006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:47.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.003+0000 7f0392000640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 0x7f037006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f037c009560 tx=0x7f037c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:47.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.007+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f038407a7c0 con 0x7f03940b5ca0 2024-04-26T20:26:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.260+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f039408ed80 con 0x7f03940b5ca0 2024-04-26T20:26:47.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.261+0000 7f038aa00640 1 -- 172.21.15.43:0/2452348322 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f038407a0c0 con 0x7f03940b5ca0 2024-04-26T20:26:47.261 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 msgr2=0x7f037006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 0x7f037006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f037c009560 tx=0x7f037c008040 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 msgr2=0x7f03940cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f03840046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 shutdown_connections 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f037006c8f0 0x7f037006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03940d3480 0x7f03940a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f03940d1d90 0x7f03940cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 --2- 172.21.15.43:0/2452348322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03940b5ca0 0x7f03940cbd20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 >> 172.21.15.43:0/2452348322 conn(0x7f03940d4c50 msgr2=0x7f03940caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 shutdown_connections 2024-04-26T20:26:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:47.264+0000 7f0393e00640 1 -- 172.21.15.43:0/2452348322 wait complete. 2024-04-26T20:26:47.284 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:26:47.284 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:26:47.284 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-04-26T20:26:47.284 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:26:47.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:47.285 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:26:47.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:26:47.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:26:47.285 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:26:47.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:47.285 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:47.343 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42557sec preferred_lft 42557sec 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-04-26T20:26:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:47.345 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:26:47.345 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:47.345 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:47.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:26:47.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr flush dev ceph-brx 2024-04-26T20:26:47.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set ceph-brx up 2024-04-26T20:26:47.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:26:47.346 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:47.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:47.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:47.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:47.812 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:26:47.878 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-04-26T20:26:47.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:47.882 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-04-26T20:26:47.940 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-04-26T20:26:47.940 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:26:47.940 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:26:47.940 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:26:47.941 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:26:47.941 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:47.942 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:47.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:26:47.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:26:47.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:26:47.942 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:48.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:48.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:48.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:48.677 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-04-26T20:26:48.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:48.730 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-04-26T20:26:48.782 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:48.782 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:48.783 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:48.783 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T20:26:48.783 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:48.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:48.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:48.973 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T20:26:48.973 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:48.973 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:48.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T20:26:48.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:26:48.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T20:26:48.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T20:26:48.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T20:26:48.974 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:49.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:49.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:49.262 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:49.263 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:49.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-04-26T20:26:49.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:26:49.263 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:49.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:49.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:49.458 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:26:49.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:49.458 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:49.500 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:26:49.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:49.500 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:49.593 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-04-26T20:26:49.677 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-04-26T20:26:49.726 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/selinux 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:26:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:26:49.728 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:26:49.728 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:26:49.728 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-04-26T20:26:49.728 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:49.728 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:49.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:49.729 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-04-26T20:26:49.773 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:26:49.773 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:26:49.820 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-04-26T20:26:49.893 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-04-26T20:26:49.934 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/selinux 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:49.935 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:49.936 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:49.938 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-04-26T20:26:50.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:2024-04-26T20:26:50.010+0000 7f1b5ad21480 -1 init, newargv = 0x560a8e4a21b0 newargc=15 2024-04-26T20:26:50.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[39221]: starting ceph client 2024-04-26T20:26:50.023 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi164.stderr:ceph-fuse[39221]: starting fuse 2024-04-26T20:26:50.025 INFO:teuthology.orchestra.run.smithi164.stdout:41 2024-04-26T20:26:50.026 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> import re 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2024-04-26T20:26:50.026 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> def _find_admin_socket(client_name): 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> contents = proc_f.read() 2024-04-26T20:26:50.027 DEBUG:teuthology.orchestra.run.smithi164:> if mountpoint in contents: 2024-04-26T20:26:50.028 DEBUG:teuthology.orchestra.run.smithi164:> return f 2024-04-26T20:26:50.028 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:26:50.028 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:50.028 DEBUG:teuthology.orchestra.run.smithi164:> print(_find_admin_socket("client.1")) 2024-04-26T20:26:50.028 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-04-26T20:26:50.214 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.1.39221.asok 2024-04-26T20:26:50.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:50.231 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39221.asok 2024-04-26T20:26:50.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:50.231 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39221.asok status 2024-04-26T20:26:50.460 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "1", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "39221", 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4550, 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2024-04-26T20:26:50.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 4550 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3187514863 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 3187514863 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.4550 192.168.0.1:0/3187514863", 2024-04-26T20:26:50.462 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "192.168.0.1:0/3187514863", 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 9, 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 26, 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "blocklisted": false, 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2024-04-26T20:26:50.463 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T20:26:50.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:26:50.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.695+0000 7f53d6800640 1 Processor -- start 2024-04-26T20:26:50.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.696+0000 7f53d6800640 1 -- start start 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 0x7f53d00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53d00d1d90 0x7f53d0162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00b5ca0 0x7f53d01626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f53d00d3090 con 0x7f53d00d1d90 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f53d00d5b50 con 0x7f53d00d3480 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.697+0000 7f53d6800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53d00cf860 con 0x7f53d00b5ca0 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53d00d1d90 0x7f53d0162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53d00d1d90 0x7f53d0162170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37118/0 (socket says 172.21.15.43:37118) 2024-04-26T20:26:50.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 -- 172.21.15.43:0/502698139 learned_addr learned my addr 172.21.15.43:0/502698139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d5e00640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00b5ca0 0x7f53d01626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d5400640 1 --2- 172.21.15.43:0/502698139 >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 0x7f53d00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 -- 172.21.15.43:0/502698139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00b5ca0 msgr2=0x7f53d01626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00b5ca0 0x7f53d01626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 -- 172.21.15.43:0/502698139 >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 msgr2=0x7f53d00c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 --2- 172.21.15.43:0/502698139 >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 0x7f53d00c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d4a00640 1 -- 172.21.15.43:0/502698139 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d00cfb00 con 0x7f53d00d1d90 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.698+0000 7f53d5400640 1 --2- 172.21.15.43:0/502698139 >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 0x7f53d00c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.699+0000 7f53d4a00640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d0162170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f53d0068d60 tx=0x7f53c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ba057b0695e68d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53cfe00640 1 -- 172.21.15.43:0/502698139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0036070 con 0x7f53d00d1d90 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53cfe00640 1 -- 172.21.15.43:0/502698139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c0031600 con 0x7f53d00d1d90 2024-04-26T20:26:50.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53cfe00640 1 -- 172.21.15.43:0/502698139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c00319d0 con 0x7f53d00d1d90 2024-04-26T20:26:50.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 -- 172.21.15.43:0/502698139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 msgr2=0x7f53d0162170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:50.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d0162170 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f53d0068d60 tx=0x7f53c002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 -- 172.21.15.43:0/502698139 shutdown_connections 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00b5ca0 0x7f53d01626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 --2- 172.21.15.43:0/502698139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d0162170 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 --2- 172.21.15.43:0/502698139 >> v2:172.21.15.164:3300/0 conn(0x7f53d00d3480 0x7f53d00c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 -- 172.21.15.43:0/502698139 >> 172.21.15.43:0/502698139 conn(0x7f53d00d4c50 msgr2=0x7f53d00d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 -- 172.21.15.43:0/502698139 shutdown_connections 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.700+0000 7f53d6800640 1 -- 172.21.15.43:0/502698139 wait complete. 2024-04-26T20:26:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.702+0000 7f53d6800640 1 Processor -- start 2024-04-26T20:26:50.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.702+0000 7f53d6800640 1 -- start start 2024-04-26T20:26:50.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 0x7f53d00b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00d3480 0x7f53d006df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53d00d5770 con 0x7f53d00d1d90 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53d00d2db0 con 0x7f53d00b5ca0 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.703+0000 7f53d6800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53d00d58f0 con 0x7f53d00d3480 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37120/0 (socket says 172.21.15.43:37120) 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 -- 172.21.15.43:0/1567416917 learned_addr learned my addr 172.21.15.43:0/1567416917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d5400640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 0x7f53d00b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 -- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00d3480 msgr2=0x7f53d006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00d3480 0x7f53d006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 -- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 msgr2=0x7f53d00b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 0x7f53d00b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d4a00640 1 -- 172.21.15.43:0/1567416917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d006c9c0 con 0x7f53d00d1d90 2024-04-26T20:26:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.704+0000 7f53d5400640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 0x7f53d00b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:50.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53d4a00640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53c0034040 tx=0x7f53c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c003f020 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c0037040 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0036070 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d009eaf0 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.705+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d006cf40 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.706+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d0069350 con 0x7f53d00d1d90 2024-04-26T20:26:50.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.707+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53c0042020 con 0x7f53d00d1d90 2024-04-26T20:26:50.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.708+0000 7f53cd600640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:50.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.709+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f53c00ac490 con 0x7f53d00d1d90 2024-04-26T20:26:50.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.709+0000 7f53d5400640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:50.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.710+0000 7f53d5400640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006fd0 tx=0x7f53c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:50.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.713+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53c0049030 con 0x7f53d00d1d90 2024-04-26T20:26:50.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.962+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53d008ed80 con 0x7f53d00d1d90 2024-04-26T20:26:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.962+0000 7f53cd600640 1 -- 172.21.15.43:0/1567416917 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f53c007a3f0 con 0x7f53d00d1d90 2024-04-26T20:26:50.963 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:26:50.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 msgr2=0x7f53b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:50.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 0x7f53b406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006fd0 tx=0x7f53c4008040 comp rx=0 tx=0).stop 2024-04-26T20:26:50.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 msgr2=0x7f53d006da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:50.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53c0034040 tx=0x7f53c0004a40 comp rx=0 tx=0).stop 2024-04-26T20:26:50.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 shutdown_connections 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f53b406c8f0 0x7f53b406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53d00d3480 0x7f53d006df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d00d1d90 0x7f53d006da20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 --2- 172.21.15.43:0/1567416917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53d00b5ca0 0x7f53d00b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.965+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 >> 172.21.15.43:0/1567416917 conn(0x7f53d00d4c50 msgr2=0x7f53d015f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.966+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 shutdown_connections 2024-04-26T20:26:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:50.966+0000 7f53d6800640 1 -- 172.21.15.43:0/1567416917 wait complete. 2024-04-26T20:26:50.986 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:26:50.987 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:26:50.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:50.987 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 6a:85:66:fc:a9:62 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:26:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::bcd3:48ff:fe5b:b704/64 scope link 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 6a:85:66:fc:a9:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::6885:66ff:fefc:a962/64 scope link 2024-04-26T20:26:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.047 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:26:51.125 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-04-26T20:26:51.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.130 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-04-26T20:26:51.182 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-04-26T20:26:51.182 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:26:51.182 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:26:51.182 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:26:51.183 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:26:51.183 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:51.183 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:51.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:26:51.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:26:51.184 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:26:51.184 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:51.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:51.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:51.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.397 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-04-26T20:26:51.441 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:26:51.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.442 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-04-26T20:26:51.536 INFO:teuthology.orchestra.run.smithi164.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T20:26:51.537 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:51.538 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:51.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:51.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T20:26:51.538 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:51.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:51.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:51.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0a:a7:4a:19:56:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.806 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::8a7:4aff:fe19:5639/64 scope link 2024-04-26T20:26:51.807 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:51.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0a:a7:4a:19:56:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:26:51.895 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:26:51.896 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.896 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::8a7:4aff:fe19:5639/64 scope link 2024-04-26T20:26:51.896 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:26:51.899 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T20:26:51.899 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T20:26:51.900 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:52.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:52.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:52.180 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:26:52.180 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-04-26T20:26:52.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.1 up 2024-04-26T20:26:52.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:26:52.180 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-04-26T20:26:52.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:26:52.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:52.380 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-26T20:26:52.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:52.381 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:52.424 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:26:52.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:26:52.425 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:52.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-04-26T20:26:52.599 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:52.653 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-04-26T20:26:52.654 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/selinux 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:52.655 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:52.656 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:52.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:52.657 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-04-26T20:26:52.706 INFO:teuthology.orchestra.run.smithi164.stdout:41 2024-04-26T20:26:52.707 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-26T20:26:52.707 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-26T20:26:52.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-04-26T20:26:52.784 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:26:52.826 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-04-26T20:26:52.826 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-04-26T20:26:52.826 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/selinux 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-04-26T20:26:52.827 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:26:52.828 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:52.829 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:26:52.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:52.830 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-04-26T20:26:52.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi164.stderr:2024-04-26T20:26:52.890+0000 7f6367363480 -1 init, newargv = 0x56107323c7b0 newargc=15 2024-04-26T20:26:52.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi164.stderr:ceph-fuse[39677]: starting ceph client 2024-04-26T20:26:52.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi164.stderr:ceph-fuse[39677]: starting fuse 2024-04-26T20:26:52.916 INFO:teuthology.orchestra.run.smithi164.stdout:41 2024-04-26T20:26:52.916 INFO:teuthology.orchestra.run.smithi164.stdout:43 2024-04-26T20:26:52.916 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-26T20:26:52.916 DEBUG:teuthology.orchestra.run.smithi164:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:26:52.916 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2024-04-26T20:26:52.916 DEBUG:teuthology.orchestra.run.smithi164:> import re 2024-04-26T20:26:52.916 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> def _find_admin_socket(client_name): 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:26:52.917 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> contents = proc_f.read() 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> if mountpoint in contents: 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> return f 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> print(_find_admin_socket("client.3")) 2024-04-26T20:26:52.918 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-04-26T20:26:53.105 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.3.39677.asok 2024-04-26T20:26:53.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-26T20:26:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:26:53.122 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39677.asok 2024-04-26T20:26:53.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39677.asok status 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "3", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "39677", 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2024-04-26T20:26:53.350 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4726, 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 4726 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 4105809044 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:26:53.351 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 4105809044 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.4726 192.168.0.2:0/4105809044", 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "192.168.0.2:0/4105809044", 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 9, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 26, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "blocklisted": false, 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2024-04-26T20:26:53.352 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-04-26T20:26:53.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.365 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-26T20:26:53.365 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:53.426 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-26T20:26:53.426 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:53.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:26:53.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.517 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-26T20:26:53.517 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:53.575 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-26T20:26:53.575 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:53.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:26:53.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.669 DEBUG:teuthology.orchestra.run.smithi164:> stat --file-system '--printf=%T 2024-04-26T20:26:53.670 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:53.727 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2024-04-26T20:26:53.727 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:53.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.728 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:26:53.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.813 DEBUG:teuthology.orchestra.run.smithi164:> stat --file-system '--printf=%T 2024-04-26T20:26:53.813 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:53.862 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2024-04-26T20:26:53.862 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:53.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:26:53.862 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:26:53.941 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T20:26:53.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:54.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.181+0000 7fbe67200640 1 Processor -- start 2024-04-26T20:26:54.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.181+0000 7fbe67200640 1 -- start start 2024-04-26T20:26:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe67200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbe601628e0 0x7fbe60162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe67200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe60145710 con 0x7fbe60145330 2024-04-26T20:26:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe67200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe60073dc0 con 0x7fbe600af720 2024-04-26T20:26:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe67200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbe600b0250 con 0x7fbe601628e0 2024-04-26T20:26:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe65e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe60145330 0x7fbe600b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37260/0 (socket says 172.21.15.43:37260) 2024-04-26T20:26:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.183+0000 7fbe65e00640 1 -- 172.21.15.43:0/1045438707 learned_addr learned my addr 172.21.15.43:0/1045438707 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770827244 0 0) 0x7fbe60145710 con 0x7fbe60145330 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe60145330 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe65400640 1 --2- 172.21.15.43:0/1045438707 >> v2:172.21.15.164:3300/0 conn(0x7fbe601628e0 0x7fbe60162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3345542612 0 0) 0x7fbe48003520 con 0x7fbe60145330 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 >> v2:172.21.15.164:3300/0 conn(0x7fbe601628e0 msgr2=0x7fbe60162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 --2- 172.21.15.43:0/1045438707 >> v2:172.21.15.164:3300/0 conn(0x7fbe601628e0 0x7fbe60162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 >> v1:172.21.15.164:6789/0 conn(0x7fbe600af720 legacy=0x7fbe600afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe600ae960 con 0x7fbe60145330 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fbe54001670 con 0x7fbe60145330 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 >> v1:172.21.15.43:6789/0 conn(0x7fbe60145330 legacy=0x7fbe600b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe48003a00 0x7fbe48003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe48005230 0x7fbe4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600b0250 con 0x7fbe48004800 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4800c190 con 0x7fbe48003a00 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.184+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4800c390 con 0x7fbe48005230 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65e00640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe48003a00 0x7fbe48003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe66800640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe48005230 0x7fbe4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 -- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe48005230 msgr2=0x7fbe4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe48005230 0x7fbe4800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 -- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe48003a00 msgr2=0x7fbe48003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:54.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe48003a00 0x7fbe48003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe65400640 1 -- 172.21.15.43:0/1045438707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe48003520 con 0x7fbe48004800 2024-04-26T20:26:54.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.185+0000 7fbe66800640 1 -- 172.21.15.43:0/1045438707 reap_dead start 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.186+0000 7fbe65400640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ba30 tx=0x7fbe48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eacc29a85b063c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50030070 con 0x7fbe48004800 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5002ae00 con 0x7fbe48004800 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe64a00640 1 -- 172.21.15.43:0/1045438707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000b600 con 0x7fbe48004800 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 -- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 msgr2=0x7fbe48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ba30 tx=0x7fbe48005830 comp rx=0 tx=0).stop 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 -- 172.21.15.43:0/1045438707 shutdown_connections 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 --2- 172.21.15.43:0/1045438707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe48004c80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 -- 172.21.15.43:0/1045438707 >> 172.21.15.43:0/1045438707 conn(0x7fbe60068090 msgr2=0x7fbe60068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:54.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.187+0000 7fbe67200640 1 -- 172.21.15.43:0/1045438707 shutdown_connections 2024-04-26T20:26:54.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.188+0000 7fbe67200640 1 -- 172.21.15.43:0/1045438707 wait complete. 2024-04-26T20:26:54.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.190+0000 7fbe67200640 1 Processor -- start 2024-04-26T20:26:54.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.190+0000 7fbe67200640 1 -- start start 2024-04-26T20:26:54.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe601f55b0 0x7fbe600ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 0x7fbe601607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600af200 con 0x7fbe48004800 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600aef20 con 0x7fbe60160340 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe67200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbe60162d70 con 0x7fbe601f55b0 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37136/0 (socket says 172.21.15.43:37136) 2024-04-26T20:26:54.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.191+0000 7fbe65e00640 1 -- 172.21.15.43:0/3750807282 learned_addr learned my addr 172.21.15.43:0/3750807282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe66800640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 0x7fbe601607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65400640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe601f55b0 0x7fbe600ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 -- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe601f55b0 msgr2=0x7fbe600ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe601f55b0 0x7fbe600ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 -- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 msgr2=0x7fbe601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 0x7fbe601607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 -- 172.21.15.43:0/3750807282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6015c6a0 con 0x7fbe48004800 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe66800640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 0x7fbe601607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe65e00640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e730 tx=0x7fbe5400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400c6c0 con 0x7fbe48004800 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe54018070 con 0x7fbe48004800 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400caa0 con 0x7fbe48004800 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.192+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe600cf620 con 0x7fbe48004800 2024-04-26T20:26:54.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.193+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe600cfbe0 con 0x7fbe48004800 2024-04-26T20:26:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.194+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe60069350 con 0x7fbe48004800 2024-04-26T20:26:54.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.195+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe5400cc40 con 0x7fbe48004800 2024-04-26T20:26:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.196+0000 7fbe5a000640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 0x7fbe4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.196+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fbe5408bdc0 con 0x7fbe48004800 2024-04-26T20:26:54.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.196+0000 7fbe65400640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 0x7fbe4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.197+0000 7fbe65400640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 0x7fbe4406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ba00 tx=0x7fbe50030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.201+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbe6015c6a0 con 0x7fbe48004800 2024-04-26T20:26:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.409+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe600b1620 con 0x7fbe48004800 2024-04-26T20:26:54.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.412+0000 7fbe5a000640 1 -- 172.21.15.43:0/3750807282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fbe54059d20 con 0x7fbe48004800 2024-04-26T20:26:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:54.414 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.415+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 msgr2=0x7fbe4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.415+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 0x7fbe4406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ba00 tx=0x7fbe50030000 comp rx=0 tx=0).stop 2024-04-26T20:26:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 msgr2=0x7fbe6015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e730 tx=0x7fbe5400ebf0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 shutdown_connections 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbe4406c8f0 0x7fbe4406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe601f55b0 0x7fbe600ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe60160340 0x7fbe601607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 --2- 172.21.15.43:0/3750807282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe48004800 0x7fbe6015c160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 >> 172.21.15.43:0/3750807282 conn(0x7fbe60068090 msgr2=0x7fbe600b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 shutdown_connections 2024-04-26T20:26:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.416+0000 7fbe67200640 1 -- 172.21.15.43:0/3750807282 wait complete. 2024-04-26T20:26:54.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:54.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.651+0000 7fb237200640 1 Processor -- start 2024-04-26T20:26:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.651+0000 7fb237200640 1 -- start start 2024-04-26T20:26:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.653+0000 7fb237200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb2300d31c0 0x7fb2300d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.653+0000 7fb237200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.653+0000 7fb237200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2300d3ae0 con 0x7fb2300b4540 2024-04-26T20:26:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.653+0000 7fb237200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb2300d01c0 con 0x7fb2300d31c0 2024-04-26T20:26:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.653+0000 7fb237200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300d2250 con 0x7fb23015f240 2024-04-26T20:26:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.654+0000 7fb235400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.654+0000 7fb235400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:47964/0 (socket says 172.21.15.43:47964) 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.654+0000 7fb235400640 1 -- 172.21.15.43:0/3631939311 learned_addr learned my addr 172.21.15.43:0/3631939311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.654+0000 7fb235e00640 1 --2- 172.21.15.43:0/3631939311 >> v2:172.21.15.164:3300/0 conn(0x7fb2300d31c0 0x7fb2300d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.655+0000 7fb235400640 1 -- 172.21.15.43:0/3631939311 >> v2:172.21.15.164:3300/0 conn(0x7fb2300d31c0 msgr2=0x7fb2300d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.655+0000 7fb235400640 1 --2- 172.21.15.43:0/3631939311 >> v2:172.21.15.164:3300/0 conn(0x7fb2300d31c0 0x7fb2300d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.655+0000 7fb235400640 1 -- 172.21.15.43:0/3631939311 >> v1:172.21.15.164:6789/0 conn(0x7fb2300b4540 legacy=0x7fb2300d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.655+0000 7fb235400640 1 -- 172.21.15.43:0/3631939311 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300d2530 con 0x7fb23015f240 2024-04-26T20:26:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.656+0000 7fb235400640 1 --2- 172.21.15.43:0/3631939311 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0e35b80b9b0b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.656+0000 7fb234a00640 1 -- 172.21.15.43:0/3631939311 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220037070 con 0x7fb23015f240 2024-04-26T20:26:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb234a00640 1 -- 172.21.15.43:0/3631939311 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb220032440 con 0x7fb23015f240 2024-04-26T20:26:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb234a00640 1 -- 172.21.15.43:0/3631939311 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220032810 con 0x7fb23015f240 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 -- 172.21.15.43:0/3631939311 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 msgr2=0x7fb2300d35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 --2- 172.21.15.43:0/3631939311 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002a270 comp rx=0 tx=0).stop 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 -- 172.21.15.43:0/3631939311 shutdown_connections 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 --2- 172.21.15.43:0/3631939311 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb23015f240 0x7fb2300d35a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 --2- 172.21.15.43:0/3631939311 >> v2:172.21.15.164:3300/0 conn(0x7fb2300d31c0 0x7fb2300d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 -- 172.21.15.43:0/3631939311 >> 172.21.15.43:0/3631939311 conn(0x7fb2300d4be0 msgr2=0x7fb2300d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 -- 172.21.15.43:0/3631939311 shutdown_connections 2024-04-26T20:26:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.657+0000 7fb237200640 1 -- 172.21.15.43:0/3631939311 wait complete. 2024-04-26T20:26:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.659+0000 7fb237200640 1 Processor -- start 2024-04-26T20:26:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.659+0000 7fb237200640 1 -- start start 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2300b4540 0x7fb230155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2300d31c0 0x7fb23015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300d1e70 con 0x7fb2301f52b0 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300cfe50 con 0x7fb2300b4540 2024-04-26T20:26:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.660+0000 7fb237200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300d1ff0 con 0x7fb2300d31c0 2024-04-26T20:26:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37138/0 (socket says 172.21.15.43:37138) 2024-04-26T20:26:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 -- 172.21.15.43:0/2654546233 learned_addr learned my addr 172.21.15.43:0/2654546233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb235e00640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2300b4540 0x7fb230155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb235400640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2300d31c0 0x7fb23015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 -- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2300d31c0 msgr2=0x7fb23015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2300d31c0 0x7fb23015f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 -- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2300b4540 msgr2=0x7fb230155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2300b4540 0x7fb230155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.661+0000 7fb236800640 1 -- 172.21.15.43:0/2654546233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2301f57b0 con 0x7fb2301f52b0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb236800640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c7a0 tx=0x7fb22c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c003b70 con 0x7fb2301f52b0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22c003d30 con 0x7fb2301f52b0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00aa90 con 0x7fb2301f52b0 2024-04-26T20:26:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301f5a90 con 0x7fb2301f52b0 2024-04-26T20:26:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.662+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301f6050 con 0x7fb2301f52b0 2024-04-26T20:26:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.663+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2300506c0 con 0x7fb2301f52b0 2024-04-26T20:26:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.664+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb22c00ac30 con 0x7fb2301f52b0 2024-04-26T20:26:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.665+0000 7fb22a000640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 0x7fb21406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.665+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fb22c08bf30 con 0x7fb2301f52b0 2024-04-26T20:26:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.666+0000 7fb235e00640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 0x7fb21406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.666+0000 7fb235e00640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 0x7fb21406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.670+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb22c059f20 con 0x7fb2301f52b0 2024-04-26T20:26:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.873+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2300b4040 con 0x7fb2301f52b0 2024-04-26T20:26:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.874+0000 7fb22a000640 1 -- 172.21.15.43:0/2654546233 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb22c059820 con 0x7fb2301f52b0 2024-04-26T20:26:54.875 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 msgr2=0x7fb21406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 0x7fb21406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb224006fd0 tx=0x7fb224008040 comp rx=0 tx=0).stop 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 msgr2=0x7fb2300ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c7a0 tx=0x7fb22c00cc60 comp rx=0 tx=0).stop 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 shutdown_connections 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb21406c8f0 0x7fb21406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb2300d31c0 0x7fb23015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2301f52b0 0x7fb2300ca5d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 --2- 172.21.15.43:0/2654546233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb2300b4540 0x7fb230155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:54.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.877+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 >> 172.21.15.43:0/2654546233 conn(0x7fb2300d4be0 msgr2=0x7fb23006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:54.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.878+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 shutdown_connections 2024-04-26T20:26:54.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:54.878+0000 7fb237200640 1 -- 172.21.15.43:0/2654546233 wait complete. 2024-04-26T20:26:54.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.114+0000 7f94b0a00640 1 Processor -- start 2024-04-26T20:26:55.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.114+0000 7f94b0a00640 1 -- start start 2024-04-26T20:26:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94b0a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94ac15a7e0 0x7f94ac0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94b0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac15d9c0 0x7f94ac0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94b0a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ac0b3900 con 0x7f94ac0b5410 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94b0a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f94ac15abc0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94b0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac0b3b70 con 0x7f94ac15d9c0 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94ab400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94ac15a7e0 0x7f94ac0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94aaa00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f94ac0b5410 0x7f94ac15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37262/0 (socket says 172.21.15.43:37262) 2024-04-26T20:26:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.116+0000 7f94aaa00640 1 -- 172.21.15.43:0/3190504285 learned_addr learned my addr 172.21.15.43:0/3190504285 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94abe00640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac15d9c0 0x7f94ac0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94ab400640 1 -- 172.21.15.43:0/3190504285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac15d9c0 msgr2=0x7f94ac0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94ab400640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac15d9c0 0x7f94ac0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94ab400640 1 -- 172.21.15.43:0/3190504285 >> v1:172.21.15.43:6789/0 conn(0x7f94ac0b5410 legacy=0x7f94ac15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94ab400640 1 -- 172.21.15.43:0/3190504285 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac0b3e50 con 0x7f94ac15a7e0 2024-04-26T20:26:55.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.117+0000 7f94aa000640 1 -- 172.21.15.43:0/3190504285 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 523597257 0 0) 0x7f94ac0b3900 con 0x7f94ac0b5410 2024-04-26T20:26:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.118+0000 7f94ab400640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac0b4650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f949c0097a0 tx=0x7f949c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cc2cb0f1e6da92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.118+0000 7f94aa000640 1 -- 172.21.15.43:0/3190504285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c037070 con 0x7f94ac15a7e0 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.118+0000 7f94aa000640 1 -- 172.21.15.43:0/3190504285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c032440 con 0x7f94ac15a7e0 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.118+0000 7f94aa000640 1 -- 172.21.15.43:0/3190504285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c032810 con 0x7f94ac15a7e0 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 -- 172.21.15.43:0/3190504285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 msgr2=0x7f94ac0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac0b4650 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f949c0097a0 tx=0x7f949c02a180 comp rx=0 tx=0).stop 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 -- 172.21.15.43:0/3190504285 shutdown_connections 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac15d9c0 0x7f94ac0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 --2- 172.21.15.43:0/3190504285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac0b4650 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 -- 172.21.15.43:0/3190504285 >> 172.21.15.43:0/3190504285 conn(0x7f94ac068090 msgr2=0x7f94ac0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 -- 172.21.15.43:0/3190504285 shutdown_connections 2024-04-26T20:26:55.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.119+0000 7f94b0a00640 1 -- 172.21.15.43:0/3190504285 wait complete. 2024-04-26T20:26:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.121+0000 7f94b0a00640 1 Processor -- start 2024-04-26T20:26:55.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.121+0000 7f94b0a00640 1 -- start start 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac0b5410 0x7f94ac13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac15d9c0 0x7f94ac15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac15e010 con 0x7f94ac15a7e0 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac15ddd0 con 0x7f94ac15d9c0 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.122+0000 7f94b0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac0b4b90 con 0x7f94ac0b5410 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37148/0 (socket says 172.21.15.43:37148) 2024-04-26T20:26:55.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 -- 172.21.15.43:0/1403562260 learned_addr learned my addr 172.21.15.43:0/1403562260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:55.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94ab400640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac0b5410 0x7f94ac13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 -- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac0b5410 msgr2=0x7f94ac13e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac0b5410 0x7f94ac13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 -- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac15d9c0 msgr2=0x7f94ac15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac15d9c0 0x7f94ac15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 -- 172.21.15.43:0/1403562260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac154490 con 0x7f94ac15a7e0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.123+0000 7f94aaa00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f949800c430 tx=0x7f949800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.124+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94980074e0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.124+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94980047e0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.124+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498004bd0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.124+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac13beb0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.124+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac13c470 con 0x7f94ac15a7e0 2024-04-26T20:26:55.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.125+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac0506c0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.126+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9498011070 con 0x7f94ac15a7e0 2024-04-26T20:26:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.128+0000 7f94a3400640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.128+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f949808bbd0 con 0x7f94ac15a7e0 2024-04-26T20:26:55.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.128+0000 7f94ab400640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.129+0000 7f94ab400640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 0x7f948c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f949c034000 tx=0x7f949c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.132+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9498059c60 con 0x7f94ac15a7e0 2024-04-26T20:26:55.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.335+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94ac13c640 con 0x7f94ac15a7e0 2024-04-26T20:26:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.336+0000 7f94a3400640 1 -- 172.21.15.43:0/1403562260 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f9498059560 con 0x7f94ac15a7e0 2024-04-26T20:26:55.337 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 msgr2=0x7f948c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 0x7f948c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f949c034000 tx=0x7f949c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 msgr2=0x7f94ac15f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f949800c430 tx=0x7f949800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 shutdown_connections 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac15d9c0 0x7f94ac15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94ac15a7e0 0x7f94ac15f270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 --2- 172.21.15.43:0/1403562260 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94ac0b5410 0x7f94ac13e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.339+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 >> 172.21.15.43:0/1403562260 conn(0x7f94ac068090 msgr2=0x7f94ac073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.340+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 shutdown_connections 2024-04-26T20:26:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.340+0000 7f94b0a00640 1 -- 172.21.15.43:0/1403562260 wait complete. 2024-04-26T20:26:55.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:55.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.559+0000 7fec47400640 1 Processor -- start 2024-04-26T20:26:55.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.560+0000 7fec47400640 1 -- start start 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 0x7fec40072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec4015d5e0 0x7fec400728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fec40146160 con 0x7fec401468c0 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fec4006dfb0 con 0x7fec4015d5e0 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.561+0000 7fec47400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fec40074180 con 0x7fec40145d80 2024-04-26T20:26:55.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.562+0000 7fec45400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.562+0000 7fec45400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37160/0 (socket says 172.21.15.43:37160) 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.562+0000 7fec44a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 0x7fec40072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.562+0000 7fec45400640 1 -- 172.21.15.43:0/86983291 learned_addr learned my addr 172.21.15.43:0/86983291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.562+0000 7fec3fe00640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.164:3300/0 conn(0x7fec4015d5e0 0x7fec400728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec44a00640 1 -- 172.21.15.43:0/86983291 >> v2:172.21.15.164:3300/0 conn(0x7fec4015d5e0 msgr2=0x7fec400728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec44a00640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.164:3300/0 conn(0x7fec4015d5e0 0x7fec400728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec44a00640 1 -- 172.21.15.43:0/86983291 >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 msgr2=0x7fec4006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec44a00640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec44a00640 1 -- 172.21.15.43:0/86983291 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec400d0710 con 0x7fec40145d80 2024-04-26T20:26:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.563+0000 7fec45400640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:26:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.564+0000 7fec44a00640 1 --2- 172.21.15.43:0/86983291 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 0x7fec40072380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec40069050 tx=0x7fec3402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e36e986d5810b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.565+0000 7fec3f400640 1 -- 172.21.15.43:0/86983291 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec40145d80 2024-04-26T20:26:55.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.565+0000 7fec3f400640 1 -- 172.21.15.43:0/86983291 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3402add0 con 0x7fec40145d80 2024-04-26T20:26:55.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.565+0000 7fec3f400640 1 -- 172.21.15.43:0/86983291 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34032660 con 0x7fec40145d80 2024-04-26T20:26:55.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 -- 172.21.15.43:0/86983291 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 msgr2=0x7fec40072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 --2- 172.21.15.43:0/86983291 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 0x7fec40072380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec40069050 tx=0x7fec3402a080 comp rx=0 tx=0).stop 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 -- 172.21.15.43:0/86983291 shutdown_connections 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.43:3300/0 conn(0x7fec401468c0 0x7fec4006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 --2- 172.21.15.43:0/86983291 >> v2:172.21.15.164:3300/0 conn(0x7fec4015d5e0 0x7fec400728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 --2- 172.21.15.43:0/86983291 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec40145d80 0x7fec40072380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 -- 172.21.15.43:0/86983291 >> 172.21.15.43:0/86983291 conn(0x7fec400b2900 msgr2=0x7fec400b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 -- 172.21.15.43:0/86983291 shutdown_connections 2024-04-26T20:26:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.566+0000 7fec47400640 1 -- 172.21.15.43:0/86983291 wait complete. 2024-04-26T20:26:55.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.568+0000 7fec47400640 1 Processor -- start 2024-04-26T20:26:55.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.568+0000 7fec47400640 1 -- start start 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec401468c0 0x7fec400b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 0x7fec400b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec400d09f0 con 0x7fec40145d80 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec4006e530 con 0x7fec4015d5e0 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec47400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fec40146370 con 0x7fec401468c0 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec44a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec44a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37168/0 (socket says 172.21.15.43:37168) 2024-04-26T20:26:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.569+0000 7fec44a00640 1 -- 172.21.15.43:0/3862829487 learned_addr learned my addr 172.21.15.43:0/3862829487 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec45400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 0x7fec400b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec3fe00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec401468c0 0x7fec400b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 -- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec401468c0 msgr2=0x7fec400b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec401468c0 0x7fec400b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 -- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 msgr2=0x7fec400b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 0x7fec400b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 -- 172.21.15.43:0/3862829487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4015bf90 con 0x7fec40145d80 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec45400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 0x7fec400b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec44a00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec3402a670 tx=0x7fec340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34040020 con 0x7fec40145d80 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34038040 con 0x7fec40145d80 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.570+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec40156670 con 0x7fec40145d80 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.571+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec40145d80 2024-04-26T20:26:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.571+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec40156c30 con 0x7fec40145d80 2024-04-26T20:26:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.572+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec40069350 con 0x7fec40145d80 2024-04-26T20:26:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.573+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec34043020 con 0x7fec40145d80 2024-04-26T20:26:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.574+0000 7fec3cc00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 0x7fec2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.574+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fec340ac410 con 0x7fec40145d80 2024-04-26T20:26:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.574+0000 7fec3fe00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 0x7fec2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.575+0000 7fec3fe00640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 0x7fec2006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec40142bb0 tx=0x7fec2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.578+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec3404a020 con 0x7fec40145d80 2024-04-26T20:26:55.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.782+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec40072800 con 0x7fec40145d80 2024-04-26T20:26:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.783+0000 7fec3cc00640 1 -- 172.21.15.43:0/3862829487 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fec34080030 con 0x7fec40145d80 2024-04-26T20:26:55.784 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.785+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 msgr2=0x7fec2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 0x7fec2006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec40142bb0 tx=0x7fec2c008040 comp rx=0 tx=0).stop 2024-04-26T20:26:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 msgr2=0x7fec4015ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec3402a670 tx=0x7fec340046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 shutdown_connections 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fec2006c8f0 0x7fec2006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec401468c0 0x7fec400b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec4015d5e0 0x7fec400b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 --2- 172.21.15.43:0/3862829487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec40145d80 0x7fec4015ba50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 >> 172.21.15.43:0/3862829487 conn(0x7fec400b2900 msgr2=0x7fec40073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 shutdown_connections 2024-04-26T20:26:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:55.786+0000 7fec47400640 1 -- 172.21.15.43:0/3862829487 wait complete. 2024-04-26T20:26:55.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.027+0000 7fafdde00640 1 Processor -- start 2024-04-26T20:26:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.027+0000 7fafdde00640 1 -- start start 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafdde00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd815ef30 0x7fafd80d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafdde00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fafd80d3300 0x7fafd8162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafdde00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd80d0d10 con 0x7fafd815f310 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafdde00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fafd80d3a70 con 0x7fafd80d3300 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafdde00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fafd80cfb20 con 0x7fafd815ef30 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafd7e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fafd815f310 0x7fafd80d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41092/0 (socket says 172.21.15.43:41092) 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.028+0000 7fafd7e00640 1 -- 172.21.15.43:0/3720185138 learned_addr learned my addr 172.21.15.43:0/3720185138 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:56.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdd400640 1 --2- 172.21.15.43:0/3720185138 >> v2:172.21.15.43:3300/0 conn(0x7fafd80d3300 0x7fafd8162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdca00640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd815ef30 0x7fafd80d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafd7400640 1 -- 172.21.15.43:0/3720185138 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890058432 0 0) 0x7fafd80d0d10 con 0x7fafd815f310 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafd7400640 1 -- 172.21.15.43:0/3720185138 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafb8003520 con 0x7fafd815f310 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdd400640 1 -- 172.21.15.43:0/3720185138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd815ef30 msgr2=0x7fafd80d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdd400640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd815ef30 0x7fafd80d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdd400640 1 -- 172.21.15.43:0/3720185138 >> v1:172.21.15.43:6789/0 conn(0x7fafd815f310 legacy=0x7fafd80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.029+0000 7fafdd400640 1 -- 172.21.15.43:0/3720185138 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80cfe00 con 0x7fafd80d3300 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.030+0000 7fafdd400640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd80d3300 0x7fafd8162c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009d90 tx=0x7fafd002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb0da3ce0cfae0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafd7400640 1 -- 172.21.15.43:0/3720185138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0031600 con 0x7fafd80d3300 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafd7400640 1 -- 172.21.15.43:0/3720185138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd00317a0 con 0x7fafd80d3300 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafd7400640 1 -- 172.21.15.43:0/3720185138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0031b70 con 0x7fafd80d3300 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 -- 172.21.15.43:0/3720185138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd80d3300 msgr2=0x7fafd8162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd80d3300 0x7fafd8162c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009d90 tx=0x7fafd002a5d0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 -- 172.21.15.43:0/3720185138 shutdown_connections 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd80d3300 0x7fafd8162c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 --2- 172.21.15.43:0/3720185138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd815ef30 0x7fafd80d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 -- 172.21.15.43:0/3720185138 >> 172.21.15.43:0/3720185138 conn(0x7fafd80d4d20 msgr2=0x7fafd80d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 -- 172.21.15.43:0/3720185138 shutdown_connections 2024-04-26T20:26:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.031+0000 7fafdde00640 1 -- 172.21.15.43:0/3720185138 wait complete. 2024-04-26T20:26:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.033+0000 7fafdde00640 1 Processor -- start 2024-04-26T20:26:56.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.033+0000 7fafdde00640 1 -- start start 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd80d3300 0x7fafd814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd80cbdf0 0x7fafd80d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80cf7f0 con 0x7fafd815ef30 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80d3770 con 0x7fafd80d3300 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafdde00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8163140 con 0x7fafd80cbdf0 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.034+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57108/0 (socket says 172.21.15.43:57108) 2024-04-26T20:26:56.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 -- 172.21.15.43:0/3247007326 learned_addr learned my addr 172.21.15.43:0/3247007326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 -- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd80cbdf0 msgr2=0x7fafd80d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd80cbdf0 0x7fafd80d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 -- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd80d3300 msgr2=0x7fafd814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd80d3300 0x7fafd814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 -- 172.21.15.43:0/3247007326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8160910 con 0x7fafd815ef30 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd7e00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c390 tx=0x7fafc000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.035+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0017070 con 0x7fafd815ef30 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.036+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc0005ce0 con 0x7fafd815ef30 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.036+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00072f0 con 0x7fafd815ef30 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.036+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd8160bf0 con 0x7fafd815ef30 2024-04-26T20:26:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.036+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81611b0 con 0x7fafd815ef30 2024-04-26T20:26:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.037+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80506c0 con 0x7fafd815ef30 2024-04-26T20:26:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.038+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafc001a070 con 0x7fafd815ef30 2024-04-26T20:26:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.039+0000 7fafd4c00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.039+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fafc008b440 con 0x7fafd815ef30 2024-04-26T20:26:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.039+0000 7fafdca00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.040+0000 7fafdca00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 0x7fafb406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fafc80098f0 tx=0x7fafc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.043+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fafc00593a0 con 0x7fafd815ef30 2024-04-26T20:26:56.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.249+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fafd80b3a00 con 0x7fafd815ef30 2024-04-26T20:26:56.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.250+0000 7fafd4c00640 1 -- 172.21.15.43:0/3247007326 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fafc0058ca0 con 0x7fafd815ef30 2024-04-26T20:26:56.250 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.252+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 msgr2=0x7fafb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.252+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 0x7fafb406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fafc80098f0 tx=0x7fafc8008040 comp rx=0 tx=0).stop 2024-04-26T20:26:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.252+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 msgr2=0x7fafd80caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c390 tx=0x7fafc000c850 comp rx=0 tx=0).stop 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 shutdown_connections 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fafd80cbdf0 0x7fafd80d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafd815ef30 0x7fafd80caf80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 --2- 172.21.15.43:0/3247007326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafd80d3300 0x7fafd814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 >> 172.21.15.43:0/3247007326 conn(0x7fafd80d4d20 msgr2=0x7fafd806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 shutdown_connections 2024-04-26T20:26:56.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.253+0000 7fafdde00640 1 -- 172.21.15.43:0/3247007326 wait complete. 2024-04-26T20:26:56.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:56.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.460+0000 7fa517a00640 1 Processor -- start 2024-04-26T20:26:56.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.460+0000 7fa517a00640 1 -- start start 2024-04-26T20:26:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.461+0000 7fa517a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa51015f240 0x7fa5100d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa517a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5100d3b00 0x7fa5100b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa517a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5100d0ac0 con 0x7fa5100d2d10 2024-04-26T20:26:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa517a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa5100b4230 con 0x7fa51015f240 2024-04-26T20:26:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa517a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa5100d0d60 con 0x7fa5100d3b00 2024-04-26T20:26:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa515000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa51015f240 0x7fa5100d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa515a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5100d3b00 0x7fa5100b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa51015f240 0x7fa5100d59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:43458/0 (socket says 172.21.15.43:43458) 2024-04-26T20:26:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 -- 172.21.15.43:0/1782722032 learned_addr learned my addr 172.21.15.43:0/1782722032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.462+0000 7fa50fe00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa5100d2d10 0x7fa5100d33f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:60500/0 (socket says 172.21.15.43:60500) 2024-04-26T20:26:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 -- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5100d3b00 msgr2=0x7fa5100b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 --2- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5100d3b00 0x7fa5100b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 -- 172.21.15.43:0/1782722032 >> v1:172.21.15.164:6789/0 conn(0x7fa5100d2d10 legacy=0x7fa5100d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.463+0000 7fa515000640 1 -- 172.21.15.43:0/1782722032 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5100d1040 con 0x7fa51015f240 2024-04-26T20:26:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.464+0000 7fa515000640 1 --2- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa51015f240 0x7fa5100d59c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa504009870 tx=0x7fa50402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3318e46084cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.465+0000 7fa50f400640 1 -- 172.21.15.43:0/1782722032 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504037070 con 0x7fa51015f240 2024-04-26T20:26:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.465+0000 7fa50f400640 1 -- 172.21.15.43:0/1782722032 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa504032440 con 0x7fa51015f240 2024-04-26T20:26:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.465+0000 7fa50f400640 1 -- 172.21.15.43:0/1782722032 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504032810 con 0x7fa51015f240 2024-04-26T20:26:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 -- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa51015f240 msgr2=0x7fa5100d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 --2- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa51015f240 0x7fa5100d59c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa504009870 tx=0x7fa50402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 -- 172.21.15.43:0/1782722032 shutdown_connections 2024-04-26T20:26:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 --2- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5100d3b00 0x7fa5100b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 --2- 172.21.15.43:0/1782722032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa51015f240 0x7fa5100d59c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 -- 172.21.15.43:0/1782722032 >> 172.21.15.43:0/1782722032 conn(0x7fa5100d4ea0 msgr2=0x7fa5100d52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 -- 172.21.15.43:0/1782722032 shutdown_connections 2024-04-26T20:26:56.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.466+0000 7fa517a00640 1 -- 172.21.15.43:0/1782722032 wait complete. 2024-04-26T20:26:56.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.467+0000 7fa517a00640 1 Processor -- start 2024-04-26T20:26:56.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.468+0000 7fa517a00640 1 -- start start 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.468+0000 7fa517a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5100d2d10 0x7fa5100b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa517a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa517a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa51015f240 0x7fa5100a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa517a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa51015f7a0 con 0x7fa5100d3b00 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa517a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa51015f620 con 0x7fa5100d2d10 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa517a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa5100b2670 con 0x7fa51015f240 2024-04-26T20:26:56.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57122/0 (socket says 172.21.15.43:57122) 2024-04-26T20:26:56.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 -- 172.21.15.43:0/3426348224 learned_addr learned my addr 172.21.15.43:0/3426348224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 -- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa51015f240 msgr2=0x7fa5100a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa51015f240 0x7fa5100a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 -- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5100d2d10 msgr2=0x7fa5100b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa515000640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5100d2d10 0x7fa5100b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5100d2d10 0x7fa5100b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.469+0000 7fa50fe00640 1 -- 172.21.15.43:0/3426348224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5100b9e60 con 0x7fa5100d3b00 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa50fe00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c380 tx=0x7fa4fc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0074e0 con 0x7fa5100d3b00 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc007680 con 0x7fa5100d3b00 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5100ba100 con 0x7fa5100d3b00 2024-04-26T20:26:56.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004a00 con 0x7fa5100d3b00 2024-04-26T20:26:56.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.470+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51009ec20 con 0x7fa5100d3b00 2024-04-26T20:26:56.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.471+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa510069350 con 0x7fa5100d3b00 2024-04-26T20:26:56.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.472+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4fc01a020 con 0x7fa5100d3b00 2024-04-26T20:26:56.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.473+0000 7fa50cc00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.473+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fa4fc08b7f0 con 0x7fa5100d3b00 2024-04-26T20:26:56.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.473+0000 7fa515000640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.474+0000 7fa515000640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa50402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.478+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4fc059760 con 0x7fa5100d3b00 2024-04-26T20:26:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.676+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5100b5e20 con 0x7fa5100d3b00 2024-04-26T20:26:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.678+0000 7fa50cc00640 1 -- 172.21.15.43:0/3426348224 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fa4fc059060 con 0x7fa5100d3b00 2024-04-26T20:26:56.679 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:56.679 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 msgr2=0x7fa4f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 0x7fa4f006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa504034000 tx=0x7fa50402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 msgr2=0x7fa5101535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c380 tx=0x7fa4fc00c840 comp rx=0 tx=0).stop 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 shutdown_connections 2024-04-26T20:26:56.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa4f006c8f0 0x7fa4f006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa51015f240 0x7fa5100a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5100d3b00 0x7fa5101535b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.680+0000 7fa517a00640 1 --2- 172.21.15.43:0/3426348224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5100d2d10 0x7fa5100b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.681+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 >> 172.21.15.43:0/3426348224 conn(0x7fa5100d4ea0 msgr2=0x7fa51006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.681+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 shutdown_connections 2024-04-26T20:26:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.681+0000 7fa517a00640 1 -- 172.21.15.43:0/3426348224 wait complete. 2024-04-26T20:26:56.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:56.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.893+0000 7f8419600640 1 Processor -- start 2024-04-26T20:26:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.893+0000 7f8419600640 1 -- start start 2024-04-26T20:26:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8419600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f841415cca0 0x7f84140cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8419600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84140d1f40 0x7f84140d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8419600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84140d12e0 con 0x7f8414160240 2024-04-26T20:26:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8419600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f84140d0240 con 0x7f841415cca0 2024-04-26T20:26:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8419600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f84140d0520 con 0x7f84140d1f40 2024-04-26T20:26:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8412000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8414160240 0x7f84140d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:60506/0 (socket says 172.21.15.43:60506) 2024-04-26T20:26:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8412000640 1 -- 172.21.15.43:0/332567488 learned_addr learned my addr 172.21.15.43:0/332567488 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.895+0000 7f8412a00640 1 --2- 172.21.15.43:0/332567488 >> v2:172.21.15.164:3300/0 conn(0x7f841415cca0 0x7f84140cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.896+0000 7f8413400640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84140d1f40 0x7f84140d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.896+0000 7f8412a00640 1 -- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84140d1f40 msgr2=0x7f84140d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.896+0000 7f8412a00640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84140d1f40 0x7f84140d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.896+0000 7f8412a00640 1 -- 172.21.15.43:0/332567488 >> v1:172.21.15.164:6789/0 conn(0x7f8414160240 legacy=0x7f84140d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.896+0000 7f8412a00640 1 -- 172.21.15.43:0/332567488 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84140d0800 con 0x7f841415cca0 2024-04-26T20:26:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.898+0000 7f8412a00640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f841415cca0 0x7f84140cfc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f840402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63199883be730e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.898+0000 7f8411600640 1 -- 172.21.15.43:0/332567488 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404037070 con 0x7f841415cca0 2024-04-26T20:26:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.898+0000 7f8411600640 1 -- 172.21.15.43:0/332567488 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404032440 con 0x7f841415cca0 2024-04-26T20:26:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.898+0000 7f8411600640 1 -- 172.21.15.43:0/332567488 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404032810 con 0x7f841415cca0 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 -- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f841415cca0 msgr2=0x7f84140cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f841415cca0 0x7f84140cfc30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f840402a180 comp rx=0 tx=0).stop 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 -- 172.21.15.43:0/332567488 shutdown_connections 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84140d1f40 0x7f84140d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 --2- 172.21.15.43:0/332567488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f841415cca0 0x7f84140cfc30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 -- 172.21.15.43:0/332567488 >> 172.21.15.43:0/332567488 conn(0x7f84140d4e40 msgr2=0x7f84140d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 -- 172.21.15.43:0/332567488 shutdown_connections 2024-04-26T20:26:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.899+0000 7f8419600640 1 -- 172.21.15.43:0/332567488 wait complete. 2024-04-26T20:26:56.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.901+0000 7f8419600640 1 Processor -- start 2024-04-26T20:26:56.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.901+0000 7f8419600640 1 -- start start 2024-04-26T20:26:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84140d1f40 0x7f84141f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 0x7f84140c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84141606b0 con 0x7f841415cca0 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84140d0dc0 con 0x7f84140d1f40 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8419600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f841415d080 con 0x7f8414160240 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.902+0000 7f8412000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57124/0 (socket says 172.21.15.43:57124) 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 -- 172.21.15.43:0/2456305201 learned_addr learned my addr 172.21.15.43:0/2456305201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412a00640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84140d1f40 0x7f84141f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8413400640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 0x7f84140c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 -- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 msgr2=0x7f84140c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 0x7f84140c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 -- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84140d1f40 msgr2=0x7f84141f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84140d1f40 0x7f84141f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 -- 172.21.15.43:0/2456305201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84140b7330 con 0x7f841415cca0 2024-04-26T20:26:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8413400640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 0x7f84140c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:56.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.903+0000 7f8412000640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f840000c400 tx=0x7f840000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.904+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400017070 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.904+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400005ce0 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.904+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000072f0 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.904+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84140b6290 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.904+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84140b6850 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.905+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414069350 con 0x7f841415cca0 2024-04-26T20:26:56.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.906+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f840001a070 con 0x7f841415cca0 2024-04-26T20:26:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.907+0000 7f840aa00640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 0x7f83f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:56.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.907+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f840008b430 con 0x7f841415cca0 2024-04-26T20:26:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.907+0000 7f8412a00640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 0x7f83f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.908+0000 7f8412a00640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 0x7f83f406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8404034000 tx=0x7f840402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:56.912+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84140d12e0 con 0x7f841415cca0 2024-04-26T20:26:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.115+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84140c84d0 con 0x7f841415cca0 2024-04-26T20:26:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.116+0000 7f840aa00640 1 -- 172.21.15.43:0/2456305201 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f8400059390 con 0x7f841415cca0 2024-04-26T20:26:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 msgr2=0x7f83f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 0x7f83f406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8404034000 tx=0x7f840402e000 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 msgr2=0x7f84140b5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f840000c400 tx=0x7f840000c8c0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 shutdown_connections 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f83f406c8f0 0x7f83f406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8414160240 0x7f84140c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f841415cca0 0x7f84140b5920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 --2- 172.21.15.43:0/2456305201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84140d1f40 0x7f84141f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 >> 172.21.15.43:0/2456305201 conn(0x7f84140d4e40 msgr2=0x7f8414155260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 shutdown_connections 2024-04-26T20:26:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.119+0000 7f8419600640 1 -- 172.21.15.43:0/2456305201 wait complete. 2024-04-26T20:26:57.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:57.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:57.228 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:26:57.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.334+0000 7ff242200640 1 Processor -- start 2024-04-26T20:26:57.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.334+0000 7ff242200640 1 -- start start 2024-04-26T20:26:57.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff242200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff23c160600 0x7ff23c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff242200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 0x7ff23c0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff242200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c0ce3c0 con 0x7ff23c0cfea0 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff242200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c1609e0 con 0x7ff23c160600 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff242200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0d3ac0 con 0x7ff23c0d0840 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff23c160600 0x7ff23c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff23c160600 0x7ff23c0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57128/0 (socket says 172.21.15.43:57128) 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff23c0cfea0 0x7ff23c0d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41098/0 (socket says 172.21.15.43:41098) 2024-04-26T20:26:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23b400640 1 -- 172.21.15.43:0/1794970419 learned_addr learned my addr 172.21.15.43:0/1794970419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23b400640 1 -- 172.21.15.43:0/1794970419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 msgr2=0x7ff23c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.336+0000 7ff23aa00640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 0x7ff23c0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.337+0000 7ff23b400640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 0x7ff23c0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.337+0000 7ff23b400640 1 -- 172.21.15.43:0/1794970419 >> v1:172.21.15.43:6789/0 conn(0x7ff23c0cfea0 legacy=0x7ff23c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.337+0000 7ff23b400640 1 -- 172.21.15.43:0/1794970419 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c0ce6a0 con 0x7ff23c160600 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.337+0000 7ff23a000640 1 -- 172.21.15.43:0/1794970419 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127236020 0 0) 0x7ff23c0ce3c0 con 0x7ff23c0cfea0 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.337+0000 7ff23aa00640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 0x7ff23c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.338+0000 7ff23b400640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c0cf420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009870 tx=0x7ff22c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe687516fe39d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.338+0000 7ff23a000640 1 -- 172.21.15.43:0/1794970419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c037070 con 0x7ff23c160600 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.338+0000 7ff23a000640 1 -- 172.21.15.43:0/1794970419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22c032440 con 0x7ff23c160600 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.338+0000 7ff23a000640 1 -- 172.21.15.43:0/1794970419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c032810 con 0x7ff23c160600 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 -- 172.21.15.43:0/1794970419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 msgr2=0x7ff23c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c0cf420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009870 tx=0x7ff22c02a250 comp rx=0 tx=0).stop 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 -- 172.21.15.43:0/1794970419 shutdown_connections 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0d0840 0x7ff23c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 --2- 172.21.15.43:0/1794970419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c0cf420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 -- 172.21.15.43:0/1794970419 >> 172.21.15.43:0/1794970419 conn(0x7ff23c0d2c70 msgr2=0x7ff23c0d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 -- 172.21.15.43:0/1794970419 shutdown_connections 2024-04-26T20:26:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.339+0000 7ff242200640 1 -- 172.21.15.43:0/1794970419 wait complete. 2024-04-26T20:26:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.340+0000 7ff242200640 1 Processor -- start 2024-04-26T20:26:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 -- start start 2024-04-26T20:26:57.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0cfea0 0x7ff23c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff23c0d0840 0x7ff23c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0d3790 con 0x7ff23c160600 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0c67c0 con 0x7ff23c0d0840 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.341+0000 7ff242200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0c7c60 con 0x7ff23c0cfea0 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57134/0 (socket says 172.21.15.43:57134) 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 -- 172.21.15.43:0/3151196563 learned_addr learned my addr 172.21.15.43:0/3151196563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23b400640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0cfea0 0x7ff23c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23aa00640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff23c0d0840 0x7ff23c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 -- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0cfea0 msgr2=0x7ff23c14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0cfea0 0x7ff23c14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 -- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff23c0d0840 msgr2=0x7ff23c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff23c0d0840 0x7ff23c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.342+0000 7ff23be00640 1 -- 172.21.15.43:0/3151196563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c156aa0 con 0x7ff23c160600 2024-04-26T20:26:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff23be00640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c7d0 tx=0x7ff23400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234018070 con 0x7ff23c160600 2024-04-26T20:26:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23400a640 con 0x7ff23c160600 2024-04-26T20:26:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23400aa70 con 0x7ff23c160600 2024-04-26T20:26:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c156d80 con 0x7ff23c160600 2024-04-26T20:26:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.343+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c157340 con 0x7ff23c160600 2024-04-26T20:26:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.345+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c0d2370 con 0x7ff23c160600 2024-04-26T20:26:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.345+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff23401b070 con 0x7ff23c160600 2024-04-26T20:26:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.346+0000 7ff233400640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.347+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7ff23408b570 con 0x7ff23c160600 2024-04-26T20:26:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.347+0000 7ff23b400640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.348+0000 7ff23b400640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff22c034000 tx=0x7ff22c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.351+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff23c0d2370 con 0x7ff23c160600 2024-04-26T20:26:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.577+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff23c070310 con 0x7ff23c160600 2024-04-26T20:26:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.578+0000 7ff233400640 1 -- 172.21.15.43:0/3151196563 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff234058e80 con 0x7ff23c160600 2024-04-26T20:26:57.579 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:57.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 msgr2=0x7ff21c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff22c034000 tx=0x7ff22c02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 msgr2=0x7ff23c1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c7d0 tx=0x7ff23400cc90 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 shutdown_connections 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff23c160600 0x7ff23c1582e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff23c0d0840 0x7ff23c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 --2- 172.21.15.43:0/3151196563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff23c0cfea0 0x7ff23c14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 >> 172.21.15.43:0/3151196563 conn(0x7ff23c0d2c70 msgr2=0x7ff23c0d17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 shutdown_connections 2024-04-26T20:26:57.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.581+0000 7ff242200640 1 -- 172.21.15.43:0/3151196563 wait complete. 2024-04-26T20:26:57.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:26:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.847+0000 7f755e400640 1 Processor -- start 2024-04-26T20:26:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.847+0000 7f755e400640 1 -- start start 2024-04-26T20:26:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.848+0000 7f755e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.848+0000 7f755e400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f75580af8a0 0x7f75580afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.848+0000 7f755e400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f755815d2e0 con 0x7f75581572b0 2024-04-26T20:26:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.848+0000 7f755e400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f755815d5c0 con 0x7f75580af8a0 2024-04-26T20:26:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.848+0000 7f755e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f755815d8a0 con 0x7f75580d12e0 2024-04-26T20:26:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.849+0000 7f7557e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.849+0000 7f7557e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:48916/0 (socket says 172.21.15.43:48916) 2024-04-26T20:26:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.849+0000 7f7557e00640 1 -- 172.21.15.43:0/2217671659 learned_addr learned my addr 172.21.15.43:0/2217671659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.849+0000 7f755da00640 1 --2- 172.21.15.43:0/2217671659 >> v2:172.21.15.164:3300/0 conn(0x7f75580af8a0 0x7f75580afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557400640 1 -- 172.21.15.43:0/2217671659 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260728907 0 0) 0x7f755815d2e0 con 0x7f75581572b0 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557400640 1 -- 172.21.15.43:0/2217671659 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7540003520 con 0x7f75581572b0 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557e00640 1 -- 172.21.15.43:0/2217671659 >> v2:172.21.15.164:3300/0 conn(0x7f75580af8a0 msgr2=0x7f75580afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557e00640 1 --2- 172.21.15.43:0/2217671659 >> v2:172.21.15.164:3300/0 conn(0x7f75580af8a0 0x7f75580afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557e00640 1 -- 172.21.15.43:0/2217671659 >> v1:172.21.15.164:6789/0 conn(0x7f75581572b0 legacy=0x7f75580d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.850+0000 7f7557e00640 1 -- 172.21.15.43:0/2217671659 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755815db80 con 0x7f75580d12e0 2024-04-26T20:26:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.851+0000 7f7557e00640 1 --2- 172.21.15.43:0/2217671659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b07303a3deb24c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.852+0000 7f7557400640 1 -- 172.21.15.43:0/2217671659 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544037070 con 0x7f75580d12e0 2024-04-26T20:26:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.852+0000 7f7557400640 1 -- 172.21.15.43:0/2217671659 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544032440 con 0x7f75580d12e0 2024-04-26T20:26:57.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.852+0000 7f7557400640 1 -- 172.21.15.43:0/2217671659 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440327f0 con 0x7f75580d12e0 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.852+0000 7f755e400640 1 -- 172.21.15.43:0/2217671659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 msgr2=0x7f75580b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.852+0000 7f755e400640 1 --2- 172.21.15.43:0/2217671659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7544009870 tx=0x7f754402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 -- 172.21.15.43:0/2217671659 shutdown_connections 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 --2- 172.21.15.43:0/2217671659 >> v2:172.21.15.164:3300/0 conn(0x7f75580af8a0 0x7f75580afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 --2- 172.21.15.43:0/2217671659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75580d12e0 0x7f75580b3e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 -- 172.21.15.43:0/2217671659 >> 172.21.15.43:0/2217671659 conn(0x7f75580684e0 msgr2=0x7f75580688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:57.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 -- 172.21.15.43:0/2217671659 shutdown_connections 2024-04-26T20:26:57.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.853+0000 7f755e400640 1 -- 172.21.15.43:0/2217671659 wait complete. 2024-04-26T20:26:57.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.854+0000 7f755e400640 1 Processor -- start 2024-04-26T20:26:57.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.855+0000 7f755e400640 1 -- start start 2024-04-26T20:26:57.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.855+0000 7f755e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580af8a0 0x7f755814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.855+0000 7f755e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75581572b0 0x7f75580aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75581609f0 con 0x7f75580d12e0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7558160870 con 0x7f75580af8a0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7558160b70 con 0x7f75581572b0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57148/0 (socket says 172.21.15.43:57148) 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 -- 172.21.15.43:0/2383342458 learned_addr learned my addr 172.21.15.43:0/2383342458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755d000640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580af8a0 0x7f755814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f755da00640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75581572b0 0x7f75580aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 -- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75581572b0 msgr2=0x7f75580aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75581572b0 0x7f75580aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 -- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580af8a0 msgr2=0x7f755814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580af8a0 0x7f755814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.856+0000 7f7557e00640 1 -- 172.21.15.43:0/2383342458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580ab9c0 con 0x7f75580d12e0 2024-04-26T20:26:57.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f7557e00640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f754402a200 tx=0x7f75440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544037040 con 0x7f75580d12e0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544003bc0 con 0x7f75580d12e0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440325f0 con 0x7f75580d12e0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75580abca0 con 0x7f75580d12e0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.857+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75580ac260 con 0x7f75580d12e0 2024-04-26T20:26:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.859+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75580506c0 con 0x7f75580d12e0 2024-04-26T20:26:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.859+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f754402ad10 con 0x7f75580d12e0 2024-04-26T20:26:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.860+0000 7f7554c00640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.860+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f75440ac260 con 0x7f75580d12e0 2024-04-26T20:26:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.861+0000 7f755d000640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:57.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.861+0000 7f755d000640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 0x7f753806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f754c0015b0 tx=0x7f754c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:57.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:57.865+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f754407a260 con 0x7f75580d12e0 2024-04-26T20:26:58.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.062+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75581564e0 con 0x7f75580d12e0 2024-04-26T20:26:58.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.063+0000 7f7554c00640 1 -- 172.21.15.43:0/2383342458 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f7544079b60 con 0x7f75580d12e0 2024-04-26T20:26:58.064 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 msgr2=0x7f753806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 0x7f753806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f754c0015b0 tx=0x7f754c009040 comp rx=0 tx=0).stop 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 msgr2=0x7f755814aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f754402a200 tx=0x7f75440046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 shutdown_connections 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f753806c8f0 0x7f753806edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75581572b0 0x7f75580aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75580d12e0 0x7f755814aba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 --2- 172.21.15.43:0/2383342458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75580af8a0 0x7f755814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.065+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 >> 172.21.15.43:0/2383342458 conn(0x7f75580684e0 msgr2=0x7f75580b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.066+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 shutdown_connections 2024-04-26T20:26:58.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.066+0000 7f755e400640 1 -- 172.21.15.43:0/2383342458 wait complete. 2024-04-26T20:26:58.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:26:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.285+0000 7fb79a800640 1 Processor -- start 2024-04-26T20:26:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.286+0000 7fb79a800640 1 -- start start 2024-04-26T20:26:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.287+0000 7fb79a800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb79415f3f0 0x7fb7940b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.287+0000 7fb79a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b5940 0x7fb794162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.287+0000 7fb79a800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79415f920 con 0x7fb794155cd0 2024-04-26T20:26:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.287+0000 7fb79a800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb79415fc00 con 0x7fb79415f3f0 2024-04-26T20:26:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.287+0000 7fb79a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb79415fee0 con 0x7fb7940b5940 2024-04-26T20:26:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb799400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb794155cd0 0x7fb794072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41112/0 (socket says 172.21.15.43:41112) 2024-04-26T20:26:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb799400640 1 -- 172.21.15.43:0/1429165738 learned_addr learned my addr 172.21.15.43:0/1429165738 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:26:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb798a00640 1 --2- 172.21.15.43:0/1429165738 >> v2:172.21.15.43:3300/0 conn(0x7fb79415f3f0 0x7fb7940b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2273469748 0 0) 0x7fb79415f920 con 0x7fb794155cd0 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb77c003520 con 0x7fb794155cd0 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.288+0000 7fb799e00640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b5940 0x7fb794162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.289+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2390907685 0 0) 0x7fb77c003520 con 0x7fb794155cd0 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.289+0000 7fb798a00640 1 -- 172.21.15.43:0/1429165738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b5940 msgr2=0x7fb794162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.289+0000 7fb798a00640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b5940 0x7fb794162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.289+0000 7fb798a00640 1 -- 172.21.15.43:0/1429165738 >> v1:172.21.15.43:6789/0 conn(0x7fb794155cd0 legacy=0x7fb794072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.289+0000 7fb798a00640 1 -- 172.21.15.43:0/1429165738 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7941601c0 con 0x7fb79415f3f0 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.290+0000 7fb798a00640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79415f3f0 0x7fb7940b5400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7800097a0 tx=0x7fb78002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d183ec45120d0258 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.290+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780037070 con 0x7fb79415f3f0 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.290+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78002ac30 con 0x7fb79415f3f0 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.290+0000 7fb793e00640 1 -- 172.21.15.43:0/1429165738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788001670 con 0x7fb79415f3f0 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 -- 172.21.15.43:0/1429165738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79415f3f0 msgr2=0x7fb7940b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79415f3f0 0x7fb7940b5400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7800097a0 tx=0x7fb78002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 -- 172.21.15.43:0/1429165738 shutdown_connections 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7940b5940 0x7fb794162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 --2- 172.21.15.43:0/1429165738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb79415f3f0 0x7fb7940b5400 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 -- 172.21.15.43:0/1429165738 >> 172.21.15.43:0/1429165738 conn(0x7fb794068090 msgr2=0x7fb7940684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 -- 172.21.15.43:0/1429165738 shutdown_connections 2024-04-26T20:26:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.291+0000 7fb79a800640 1 -- 172.21.15.43:0/1429165738 wait complete. 2024-04-26T20:26:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.293+0000 7fb79a800640 1 Processor -- start 2024-04-26T20:26:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.293+0000 7fb79a800640 1 -- start start 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b5940 0x7fb79414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb79415f3f0 0x7fb7940b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7941560b0 con 0x7fb794155cd0 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb794072d10 con 0x7fb7940b5940 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb79a800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb794156230 con 0x7fb79415f3f0 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57170/0 (socket says 172.21.15.43:57170) 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 -- 172.21.15.43:0/902763046 learned_addr learned my addr 172.21.15.43:0/902763046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb799400640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b5940 0x7fb79414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb799e00640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb79415f3f0 0x7fb7940b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 -- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb79415f3f0 msgr2=0x7fb7940b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb79415f3f0 0x7fb7940b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 -- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b5940 msgr2=0x7fb79414dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b5940 0x7fb79414dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.294+0000 7fb798a00640 1 -- 172.21.15.43:0/902763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7941f8010 con 0x7fb794155cd0 2024-04-26T20:26:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb798a00640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb7800098c0 tx=0x7fb7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780037050 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780043070 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780006e10 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7941f82f0 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.295+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7941f88b0 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.296+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7940506c0 con 0x7fb794155cd0 2024-04-26T20:26:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.297+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb78003e670 con 0x7fb794155cd0 2024-04-26T20:26:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.298+0000 7fb791600640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 0x7fb77406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.298+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fb7800ac840 con 0x7fb794155cd0 2024-04-26T20:26:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.298+0000 7fb799400640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 0x7fb77406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.299+0000 7fb799400640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 0x7fb77406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb788005880 tx=0x7fb788009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.305+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb78007a8d0 con 0x7fb794155cd0 2024-04-26T20:26:58.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.507+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb7940b22d0 con 0x7fb794155cd0 2024-04-26T20:26:58.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.508+0000 7fb791600640 1 -- 172.21.15.43:0/902763046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb78007a1d0 con 0x7fb794155cd0 2024-04-26T20:26:58.509 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:58.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.510+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 msgr2=0x7fb77406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.510+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 0x7fb77406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb788005880 tx=0x7fb788009040 comp rx=0 tx=0).stop 2024-04-26T20:26:58.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.510+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 msgr2=0x7fb79414f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.510+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb7800098c0 tx=0x7fb7800046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 shutdown_connections 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb77406c8f0 0x7fb77406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb79415f3f0 0x7fb7940b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb794155cd0 0x7fb79414f2c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 --2- 172.21.15.43:0/902763046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7940b5940 0x7fb79414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 >> 172.21.15.43:0/902763046 conn(0x7fb794068090 msgr2=0x7fb794068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 shutdown_connections 2024-04-26T20:26:58.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.511+0000 7fb79a800640 1 -- 172.21.15.43:0/902763046 wait complete. 2024-04-26T20:26:58.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:26:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.734+0000 7fd2fb200640 1 Processor -- start 2024-04-26T20:26:58.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.734+0000 7fd2fb200640 1 -- start start 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2fb200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2fc072620 0x7fd2fc0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2fb200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2fb200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc072a80 con 0x7fd2fc0cd180 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2fb200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0ce720 con 0x7fd2fc072620 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2fb200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0cd560 con 0x7fd2fc0ce2b0 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2f9400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2fc072620 0x7fd2fc0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2f9400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2fc072620 0x7fd2fc0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57182/0 (socket says 172.21.15.43:57182) 2024-04-26T20:26:58.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.736+0000 7fd2f9400640 1 -- 172.21.15.43:0/1843635591 learned_addr learned my addr 172.21.15.43:0/1843635591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2fa800640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2f9400640 1 -- 172.21.15.43:0/1843635591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 msgr2=0x7fd2fc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2f9400640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2f9400640 1 -- 172.21.15.43:0/1843635591 >> v1:172.21.15.164:6789/0 conn(0x7fd2fc0cd180 legacy=0x7fd2fc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2f9400640 1 -- 172.21.15.43:0/1843635591 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc072d00 con 0x7fd2fc072620 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.737+0000 7fd2fa800640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.738+0000 7fd2f9400640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc072620 0x7fd2fc0cdd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22748b0998d6b3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.738+0000 7fd2f8a00640 1 -- 172.21.15.43:0/1843635591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8037070 con 0x7fd2fc072620 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.738+0000 7fd2f8a00640 1 -- 172.21.15.43:0/1843635591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8032440 con 0x7fd2fc072620 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.738+0000 7fd2f8a00640 1 -- 172.21.15.43:0/1843635591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8032810 con 0x7fd2fc072620 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 -- 172.21.15.43:0/1843635591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc072620 msgr2=0x7fd2fc0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc072620 0x7fd2fc0cdd70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e802a200 comp rx=0 tx=0).stop 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 -- 172.21.15.43:0/1843635591 shutdown_connections 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 --2- 172.21.15.43:0/1843635591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc072620 0x7fd2fc0cdd70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 -- 172.21.15.43:0/1843635591 >> 172.21.15.43:0/1843635591 conn(0x7fd2fc068090 msgr2=0x7fd2fc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 -- 172.21.15.43:0/1843635591 shutdown_connections 2024-04-26T20:26:58.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.739+0000 7fd2fb200640 1 -- 172.21.15.43:0/1843635591 wait complete. 2024-04-26T20:26:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.741+0000 7fd2fb200640 1 Processor -- start 2024-04-26T20:26:58.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.741+0000 7fd2fb200640 1 -- start start 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2fc072620 0x7fd2fc141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0735a0 con 0x7fd2fc0cd180 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0732c0 con 0x7fd2fc072620 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2fb200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc073da0 con 0x7fd2fc0ce2b0 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2f9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2f9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57194/0 (socket says 172.21.15.43:57194) 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2f9400640 1 -- 172.21.15.43:0/508040638 learned_addr learned my addr 172.21.15.43:0/508040638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.742+0000 7fd2f9e00640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2fc072620 0x7fd2fc141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2fa800640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 -- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 msgr2=0x7fd2fc143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 -- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2fc072620 msgr2=0x7fd2fc141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2fc072620 0x7fd2fc141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 -- 172.21.15.43:0/508040638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc140c90 con 0x7fd2fc0cd180 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2fa800640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f9400640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8041020 con 0x7fd2fc0cd180 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8030970 con 0x7fd2fc0cd180 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc140f70 con 0x7fd2fc0cd180 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.743+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8037040 con 0x7fd2fc0cd180 2024-04-26T20:26:58.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.744+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc141530 con 0x7fd2fc0cd180 2024-04-26T20:26:58.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.745+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc071e60 con 0x7fd2fc0cd180 2024-04-26T20:26:58.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.745+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2e8044020 con 0x7fd2fc0cd180 2024-04-26T20:26:58.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.747+0000 7fd2f2000640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:58.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.747+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd2e80aca10 con 0x7fd2fc0cd180 2024-04-26T20:26:58.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.747+0000 7fd2f9e00640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:58.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.748+0000 7fd2f9e00640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec004ea0 tx=0x7fd2ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:58.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.752+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2fc071e60 con 0x7fd2fc0cd180 2024-04-26T20:26:58.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.958+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2fc15cc20 con 0x7fd2fc0cd180 2024-04-26T20:26:58.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.960+0000 7fd2f2000640 1 -- 172.21.15.43:0/508040638 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fd2e804b030 con 0x7fd2fc0cd180 2024-04-26T20:26:58.961 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:26:58.961 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 msgr2=0x7fd2dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec004ea0 tx=0x7fd2ec009040 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 msgr2=0x7fd2fc15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e80046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 shutdown_connections 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd2fc0ce2b0 0x7fd2fc143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2fc0cd180 0x7fd2fc15c1c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 --2- 172.21.15.43:0/508040638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2fc072620 0x7fd2fc141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 >> 172.21.15.43:0/508040638 conn(0x7fd2fc068090 msgr2=0x7fd2fc15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 shutdown_connections 2024-04-26T20:26:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:58.963+0000 7fd2fb200640 1 -- 172.21.15.43:0/508040638 wait complete. 2024-04-26T20:26:58.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:26:58.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:26:59.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.197+0000 7fbeffe00640 1 Processor -- start 2024-04-26T20:26:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.197+0000 7fbeffe00640 1 -- start start 2024-04-26T20:26:59.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.198+0000 7fbeffe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbf000d3830 0x7fbf000d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.198+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.198+0000 7fbeffe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf000d2d70 con 0x7fbf0015f180 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.198+0000 7fbeffe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbf0015f560 con 0x7fbf000d3830 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.198+0000 7fbeffe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbf000d06e0 con 0x7fbf000b3cb0 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.199+0000 7fbeff400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.199+0000 7fbeff400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:48968/0 (socket says 172.21.15.43:48968) 2024-04-26T20:26:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.199+0000 7fbeff400640 1 -- 172.21.15.43:0/1125791001 learned_addr learned my addr 172.21.15.43:0/1125791001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.199+0000 7fbefe000640 1 --2- 172.21.15.43:0/1125791001 >> v2:172.21.15.164:3300/0 conn(0x7fbf000d3830 0x7fbf000d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbefd600640 1 -- 172.21.15.43:0/1125791001 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1280180053 0 0) 0x7fbf000d2d70 con 0x7fbf0015f180 2024-04-26T20:26:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbefd600640 1 -- 172.21.15.43:0/1125791001 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee4003520 con 0x7fbf0015f180 2024-04-26T20:26:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbeff400640 1 -- 172.21.15.43:0/1125791001 >> v2:172.21.15.164:3300/0 conn(0x7fbf000d3830 msgr2=0x7fbf000d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbeff400640 1 --2- 172.21.15.43:0/1125791001 >> v2:172.21.15.164:3300/0 conn(0x7fbf000d3830 0x7fbf000d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbeff400640 1 -- 172.21.15.43:0/1125791001 >> v1:172.21.15.164:6789/0 conn(0x7fbf0015f180 legacy=0x7fbf000d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.200+0000 7fbeff400640 1 -- 172.21.15.43:0/1125791001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf000d09c0 con 0x7fbf000b3cb0 2024-04-26T20:26:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.201+0000 7fbeff400640 1 --2- 172.21.15.43:0/1125791001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ebc52cf40d6a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbefd600640 1 -- 172.21.15.43:0/1125791001 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8037070 con 0x7fbf000b3cb0 2024-04-26T20:26:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbefd600640 1 -- 172.21.15.43:0/1125791001 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8032440 con 0x7fbf000b3cb0 2024-04-26T20:26:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbefd600640 1 -- 172.21.15.43:0/1125791001 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80327f0 con 0x7fbf000b3cb0 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 -- 172.21.15.43:0/1125791001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 msgr2=0x7fbf000b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 --2- 172.21.15.43:0/1125791001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef802a270 comp rx=0 tx=0).stop 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 -- 172.21.15.43:0/1125791001 shutdown_connections 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 --2- 172.21.15.43:0/1125791001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf000b1740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 --2- 172.21.15.43:0/1125791001 >> v2:172.21.15.164:3300/0 conn(0x7fbf000d3830 0x7fbf000d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 -- 172.21.15.43:0/1125791001 >> 172.21.15.43:0/1125791001 conn(0x7fbf000d4bd0 msgr2=0x7fbf000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 -- 172.21.15.43:0/1125791001 shutdown_connections 2024-04-26T20:26:59.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.202+0000 7fbeffe00640 1 -- 172.21.15.43:0/1125791001 wait complete. 2024-04-26T20:26:59.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.204+0000 7fbeffe00640 1 Processor -- start 2024-04-26T20:26:59.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.204+0000 7fbeffe00640 1 -- start start 2024-04-26T20:26:59.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf00153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d3830 0x7fbf000cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000d11c0 con 0x7fbf0015f180 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000d2a40 con 0x7fbf000d3830 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.205+0000 7fbeffe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbf000ce270 con 0x7fbf000b3cb0 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57202/0 (socket says 172.21.15.43:57202) 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 -- 172.21.15.43:0/599724876 learned_addr learned my addr 172.21.15.43:0/599724876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbefea00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf00153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbefe000640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d3830 0x7fbf000cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 -- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 msgr2=0x7fbf00153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf00153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 -- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d3830 msgr2=0x7fbf000cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d3830 0x7fbf000cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.206+0000 7fbeff400640 1 -- 172.21.15.43:0/599724876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf000cd660 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbeff400640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8037040 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8042070 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8003de0 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf000cd940 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.207+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf000cdf00 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.208+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf000ce3f0 con 0x7fbf0015f180 2024-04-26T20:26:59.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.209+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbef8047460 con 0x7fbf0015f180 2024-04-26T20:26:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.210+0000 7fbef6a00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.210+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fbef80ac680 con 0x7fbf0015f180 2024-04-26T20:26:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.211+0000 7fbefea00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.211+0000 7fbefea00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 0x7fbedc06ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef0001cb0 tx=0x7fbef0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.215+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbef807a670 con 0x7fbf0015f180 2024-04-26T20:26:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.428+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf000506c0 con 0x7fbf0015f180 2024-04-26T20:26:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.428+0000 7fbef6a00640 1 -- 172.21.15.43:0/599724876 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fbef8079f70 con 0x7fbf0015f180 2024-04-26T20:26:59.429 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:26:59.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 msgr2=0x7fbedc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 0x7fbedc06ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef0001cb0 tx=0x7fbef0009040 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 msgr2=0x7fbf000b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef80046c0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 shutdown_connections 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbedc06c920 0x7fbedc06ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf0015f180 0x7fbf000b6390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbf000d3830 0x7fbf000cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 --2- 172.21.15.43:0/599724876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbf000b3cb0 0x7fbf00153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.431+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 >> 172.21.15.43:0/599724876 conn(0x7fbf000d4bd0 msgr2=0x7fbf0015c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.432+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 shutdown_connections 2024-04-26T20:26:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.432+0000 7fbeffe00640 1 -- 172.21.15.43:0/599724876 wait complete. 2024-04-26T20:26:59.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:26:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.667+0000 7f64cea00640 1 Processor -- start 2024-04-26T20:26:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.667+0000 7f64cea00640 1 -- start start 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.669+0000 7f64cea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f64c815f150 0x7f64c8070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.669+0000 7f64cea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c80cd690 0x7f64c80cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.669+0000 7f64cea00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c80d19e0 con 0x7f64c80d3610 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.669+0000 7f64cea00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f64c80d0380 con 0x7f64c815f150 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.669+0000 7f64cea00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f64c80d1790 con 0x7f64c80cd690 2024-04-26T20:26:59.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f64c815f150 0x7f64c8070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f64c815f150 0x7f64c8070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57218/0 (socket says 172.21.15.43:57218) 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 -- 172.21.15.43:0/2594732330 learned_addr learned my addr 172.21.15.43:0/2594732330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64ce000640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c80cd690 0x7f64c80cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 -- 172.21.15.43:0/2594732330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c80cd690 msgr2=0x7f64c80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c80cd690 0x7f64c80cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 -- 172.21.15.43:0/2594732330 >> v1:172.21.15.164:6789/0 conn(0x7f64c80d3610 legacy=0x7f64c8070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.670+0000 7f64cd600640 1 -- 172.21.15.43:0/2594732330 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c80d1cc0 con 0x7f64c815f150 2024-04-26T20:26:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.672+0000 7f64cd600640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c815f150 0x7f64c8070130 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009870 tx=0x7f64bc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3955f5958fb21c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.672+0000 7f64c7e00640 1 -- 172.21.15.43:0/2594732330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc037070 con 0x7f64c815f150 2024-04-26T20:26:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.672+0000 7f64c7e00640 1 -- 172.21.15.43:0/2594732330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64bc02ae50 con 0x7f64c815f150 2024-04-26T20:26:59.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.672+0000 7f64c7e00640 1 -- 172.21.15.43:0/2594732330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc032860 con 0x7f64c815f150 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 -- 172.21.15.43:0/2594732330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c815f150 msgr2=0x7f64c8070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c815f150 0x7f64c8070130 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009870 tx=0x7f64bc02a300 comp rx=0 tx=0).stop 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 -- 172.21.15.43:0/2594732330 shutdown_connections 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c80cd690 0x7f64c80cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 --2- 172.21.15.43:0/2594732330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c815f150 0x7f64c8070130 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 -- 172.21.15.43:0/2594732330 >> 172.21.15.43:0/2594732330 conn(0x7f64c80d2af0 msgr2=0x7f64c80d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 -- 172.21.15.43:0/2594732330 shutdown_connections 2024-04-26T20:26:59.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.673+0000 7f64cea00640 1 -- 172.21.15.43:0/2594732330 wait complete. 2024-04-26T20:26:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.675+0000 7f64cea00640 1 Processor -- start 2024-04-26T20:26:59.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.675+0000 7f64cea00640 1 -- start start 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c80cd690 0x7f64c81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c815f150 0x7f64c81617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64c80d1fa0 con 0x7f64c80d3610 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64c80cd200 con 0x7f64c80cd690 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.676+0000 7f64cea00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f64c8070d80 con 0x7f64c815f150 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57228/0 (socket says 172.21.15.43:57228) 2024-04-26T20:26:59.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 -- 172.21.15.43:0/326966020 learned_addr learned my addr 172.21.15.43:0/326966020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:26:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ce000640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c815f150 0x7f64c81617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64cd600640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c80cd690 0x7f64c81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 -- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c815f150 msgr2=0x7f64c81617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c815f150 0x7f64c81617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 -- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c80cd690 msgr2=0x7f64c81570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c80cd690 0x7f64c81570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.677+0000 7f64ccc00640 1 -- 172.21.15.43:0/326966020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c80c7580 con 0x7f64c80d3610 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64ccc00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c400 tx=0x7f64b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40052d0 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4005490 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4007260 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c80c7860 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.678+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c80c7e20 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.679+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c80506c0 con 0x7f64c80d3610 2024-04-26T20:26:59.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.680+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64b4007400 con 0x7f64c80d3610 2024-04-26T20:26:59.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.682+0000 7f64c5600640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 0x7f64a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:26:59.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.682+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f64b408b520 con 0x7f64c80d3610 2024-04-26T20:26:59.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.682+0000 7f64cd600640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 0x7f64a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:26:59.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.683+0000 7f64cd600640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 0x7f64a806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64bc034000 tx=0x7f64bc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:26:59.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.686+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64b4059510 con 0x7f64c80d3610 2024-04-26T20:26:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.892+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64c80cef40 con 0x7f64c80d3610 2024-04-26T20:26:59.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.894+0000 7f64c5600640 1 -- 172.21.15.43:0/326966020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f64b4058e10 con 0x7f64c80d3610 2024-04-26T20:26:59.894 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 msgr2=0x7f64a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 0x7f64a806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64bc034000 tx=0x7f64bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 msgr2=0x7f64c8161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64b400c400 tx=0x7f64b400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 shutdown_connections 2024-04-26T20:26:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f64a806c8f0 0x7f64a806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f64c815f150 0x7f64c81617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64c80d3610 0x7f64c8161280 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 --2- 172.21.15.43:0/326966020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64c80cd690 0x7f64c81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.896+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 >> 172.21.15.43:0/326966020 conn(0x7f64c80d2af0 msgr2=0x7f64c80d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.897+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 shutdown_connections 2024-04-26T20:26:59.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:26:59.897+0000 7f64cea00640 1 -- 172.21.15.43:0/326966020 wait complete. 2024-04-26T20:26:59.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.134+0000 7f952fe00640 1 Processor -- start 2024-04-26T20:27:00.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.134+0000 7f952fe00640 1 -- start start 2024-04-26T20:27:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952fe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95300d0bd0 0x7f95300d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9530163bc0 con 0x7f9530163760 2024-04-26T20:27:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952fe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f953014cf90 con 0x7f95300d0bd0 2024-04-26T20:27:00.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f953006dc60 con 0x7f953014cbb0 2024-04-26T20:27:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952ea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952ea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:49006/0 (socket says 172.21.15.43:49006) 2024-04-26T20:27:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f952ea00640 1 -- 172.21.15.43:0/2765733268 learned_addr learned my addr 172.21.15.43:0/2765733268 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.136+0000 7f9527e00640 1 --1- 172.21.15.43:0/2765733268 >> v1:172.21.15.164:6789/0 conn(0x7f9530163760 0x7f95300b5bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:60540/0 (socket says 172.21.15.43:60540) 2024-04-26T20:27:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952f400640 1 --2- 172.21.15.43:0/2765733268 >> v2:172.21.15.164:3300/0 conn(0x7f95300d0bd0 0x7f95300d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952e000640 1 -- 172.21.15.43:0/2765733268 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44567189 0 0) 0x7f9518001110 con 0x7f9530163760 2024-04-26T20:27:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952ea00640 1 -- 172.21.15.43:0/2765733268 >> v2:172.21.15.164:3300/0 conn(0x7f95300d0bd0 msgr2=0x7f95300d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952ea00640 1 --2- 172.21.15.43:0/2765733268 >> v2:172.21.15.164:3300/0 conn(0x7f95300d0bd0 0x7f95300d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952ea00640 1 -- 172.21.15.43:0/2765733268 >> v1:172.21.15.164:6789/0 conn(0x7f9530163760 legacy=0x7f95300b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.137+0000 7f952ea00640 1 -- 172.21.15.43:0/2765733268 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953006df40 con 0x7f953014cbb0 2024-04-26T20:27:00.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.139+0000 7f952ea00640 1 --2- 172.21.15.43:0/2765733268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9520009870 tx=0x7f952002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb2c1175a7c2b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.139+0000 7f952e000640 1 -- 172.21.15.43:0/2765733268 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520037070 con 0x7f953014cbb0 2024-04-26T20:27:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.139+0000 7f952e000640 1 -- 172.21.15.43:0/2765733268 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520032440 con 0x7f953014cbb0 2024-04-26T20:27:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.139+0000 7f952e000640 1 -- 172.21.15.43:0/2765733268 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520032810 con 0x7f953014cbb0 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 -- 172.21.15.43:0/2765733268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 msgr2=0x7f95300b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 --2- 172.21.15.43:0/2765733268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9520009870 tx=0x7f952002a270 comp rx=0 tx=0).stop 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 -- 172.21.15.43:0/2765733268 shutdown_connections 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 --2- 172.21.15.43:0/2765733268 >> v2:172.21.15.164:3300/0 conn(0x7f95300d0bd0 0x7f95300d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 --2- 172.21.15.43:0/2765733268 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300b4320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 -- 172.21.15.43:0/2765733268 >> 172.21.15.43:0/2765733268 conn(0x7f95300684e0 msgr2=0x7f95300688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 -- 172.21.15.43:0/2765733268 shutdown_connections 2024-04-26T20:27:00.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.140+0000 7f952fe00640 1 -- 172.21.15.43:0/2765733268 wait complete. 2024-04-26T20:27:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.142+0000 7f952fe00640 1 Processor -- start 2024-04-26T20:27:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.142+0000 7f952fe00640 1 -- start start 2024-04-26T20:27:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95300d0bd0 0x7f9530162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f953006e7e0 con 0x7f9530163760 2024-04-26T20:27:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f953006e500 con 0x7f95300d0bd0 2024-04-26T20:27:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f953006ed00 con 0x7f953014cbb0 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.143+0000 7f952f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57236/0 (socket says 172.21.15.43:57236) 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 -- 172.21.15.43:0/3393370144 learned_addr learned my addr 172.21.15.43:0/3393370144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952ea00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95300d0bd0 0x7f9530162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f9527e00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 -- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 msgr2=0x7f95300ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 -- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95300d0bd0 msgr2=0x7f9530162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95300d0bd0 0x7f9530162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 -- 172.21.15.43:0/3393370144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9530145df0 con 0x7f9530163760 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.144+0000 7f952f400640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f952800efc0 tx=0x7f952800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.145+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528018020 con 0x7f9530163760 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.145+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9528014480 con 0x7f9530163760 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.145+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528014870 con 0x7f9530163760 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.145+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95301460d0 con 0x7f9530163760 2024-04-26T20:27:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.145+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9530146690 con 0x7f9530163760 2024-04-26T20:27:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.146+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95300506c0 con 0x7f9530163760 2024-04-26T20:27:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.147+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9528005d10 con 0x7f9530163760 2024-04-26T20:27:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.148+0000 7f9527400640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.149+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f952808ce40 con 0x7f9530163760 2024-04-26T20:27:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.149+0000 7f952ea00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.150+0000 7f952ea00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 0x7f950c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9520034000 tx=0x7f952002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.153+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9528022030 con 0x7f9530163760 2024-04-26T20:27:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.357+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9530068c00 con 0x7f9530163760 2024-04-26T20:27:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.358+0000 7f9527400640 1 -- 172.21.15.43:0/3393370144 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f9528010040 con 0x7f9530163760 2024-04-26T20:27:00.358 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 msgr2=0x7f950c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 0x7f950c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9520034000 tx=0x7f952002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 msgr2=0x7f95301f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f952800efc0 tx=0x7f952800c490 comp rx=0 tx=0).stop 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 shutdown_connections 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f950c06c8f0 0x7f950c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f953014cbb0 0x7f95300ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9530163760 0x7f95301f9f70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 --2- 172.21.15.43:0/3393370144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95300d0bd0 0x7f9530162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.361+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 >> 172.21.15.43:0/3393370144 conn(0x7f95300684e0 msgr2=0x7f95300b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.362+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 shutdown_connections 2024-04-26T20:27:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.362+0000 7f952fe00640 1 -- 172.21.15.43:0/3393370144 wait complete. 2024-04-26T20:27:00.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:00.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.591+0000 7f9ac4e00640 1 Processor -- start 2024-04-26T20:27:00.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.591+0000 7f9ac4e00640 1 -- start start 2024-04-26T20:27:00.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ac4e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ac015a7e0 0x7f9ac00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ac4e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac00b3900 con 0x7f9ac00b5410 2024-04-26T20:27:00.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ac4e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac00b3b70 con 0x7f9ac015d9c0 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ac4e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9ac015abc0 con 0x7f9ac015a7e0 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9abf400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ac015a7e0 0x7f9ac00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ab7e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9ac00b5410 0x7f9ac015d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41128/0 (socket says 172.21.15.43:41128) 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9abf400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ac015a7e0 0x7f9ac00b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57238/0 (socket says 172.21.15.43:57238) 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.593+0000 7f9ab7e00640 1 -- 172.21.15.43:0/2048272763 learned_addr learned my addr 172.21.15.43:0/2048272763 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.594+0000 7f9abf400640 1 -- 172.21.15.43:0/2048272763 >> v1:172.21.15.164:6789/0 conn(0x7f9ac015d9c0 legacy=0x7f9ac015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.594+0000 7f9abf400640 1 -- 172.21.15.43:0/2048272763 >> v1:172.21.15.43:6789/0 conn(0x7f9ac00b5410 legacy=0x7f9ac015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.594+0000 7f9abf400640 1 -- 172.21.15.43:0/2048272763 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac00b3e50 con 0x7f9ac015a7e0 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.594+0000 7f9abea00640 1 -- 172.21.15.43:0/2048272763 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817224565 0 0) 0x7f9ac00b3900 con 0x7f9ac00b5410 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.595+0000 7f9abf400640 1 --2- 172.21.15.43:0/2048272763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac015a7e0 0x7f9ac00b4650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097a0 tx=0x7f9ab002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d2dea418e6129e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.595+0000 7f9abea00640 1 -- 172.21.15.43:0/2048272763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0037070 con 0x7f9ac015a7e0 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.595+0000 7f9abea00640 1 -- 172.21.15.43:0/2048272763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0032440 con 0x7f9ac015a7e0 2024-04-26T20:27:00.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.595+0000 7f9abea00640 1 -- 172.21.15.43:0/2048272763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0032810 con 0x7f9ac015a7e0 2024-04-26T20:27:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 -- 172.21.15.43:0/2048272763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac015a7e0 msgr2=0x7f9ac00b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/2048272763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac015a7e0 0x7f9ac00b4650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00097a0 tx=0x7f9ab002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 -- 172.21.15.43:0/2048272763 shutdown_connections 2024-04-26T20:27:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/2048272763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac015a7e0 0x7f9ac00b4650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 -- 172.21.15.43:0/2048272763 >> 172.21.15.43:0/2048272763 conn(0x7f9ac0068090 msgr2=0x7f9ac00684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 -- 172.21.15.43:0/2048272763 shutdown_connections 2024-04-26T20:27:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.596+0000 7f9ac4e00640 1 -- 172.21.15.43:0/2048272763 wait complete. 2024-04-26T20:27:00.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.598+0000 7f9ac4e00640 1 Processor -- start 2024-04-26T20:27:00.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.598+0000 7f9ac4e00640 1 -- start start 2024-04-26T20:27:00.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ac015a7e0 0x7f9ac0156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ac015d9c0 0x7f9ac01f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac00b4b90 con 0x7f9ac00b5410 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac015e010 con 0x7f9ac015a7e0 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.599+0000 7f9ac4e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac015e9f0 con 0x7f9ac015d9c0 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57254/0 (socket says 172.21.15.43:57254) 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 -- 172.21.15.43:0/1065783250 learned_addr learned my addr 172.21.15.43:0/1065783250 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 -- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ac015d9c0 msgr2=0x7f9ac01f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ac015d9c0 0x7f9ac01f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 -- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ac015a7e0 msgr2=0x7f9ac0156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ac015a7e0 0x7f9ac0156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 -- 172.21.15.43:0/1065783250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac01f8750 con 0x7f9ac00b5410 2024-04-26T20:27:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.600+0000 7f9abf400640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00098c0 tx=0x7f9ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.601+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0041070 con 0x7f9ac00b5410 2024-04-26T20:27:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.601+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab003c870 con 0x7f9ac00b5410 2024-04-26T20:27:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.601+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0037040 con 0x7f9ac00b5410 2024-04-26T20:27:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.601+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01f8a30 con 0x7f9ac00b5410 2024-04-26T20:27:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.601+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01562f0 con 0x7f9ac00b5410 2024-04-26T20:27:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.602+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac00506c0 con 0x7f9ac00b5410 2024-04-26T20:27:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.603+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ab0044070 con 0x7f9ac00b5410 2024-04-26T20:27:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.604+0000 7f9ab7400640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 0x7f9a9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.604+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f9ab00acb70 con 0x7f9ac00b5410 2024-04-26T20:27:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.605+0000 7f9ab7e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 0x7f9a9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.605+0000 7f9ab7e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 0x7f9a9c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8007fd0 tx=0x7f9aa8007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.608+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ab007ac80 con 0x7f9ac00b5410 2024-04-26T20:27:00.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.812+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9ac0156530 con 0x7f9ac00b5410 2024-04-26T20:27:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.813+0000 7f9ab7400640 1 -- 172.21.15.43:0/1065783250 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f9ab007a580 con 0x7f9ac00b5410 2024-04-26T20:27:00.814 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 msgr2=0x7f9a9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 0x7f9a9c06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8007fd0 tx=0x7f9aa8007920 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 msgr2=0x7f9ac014c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00098c0 tx=0x7f9ab00046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 shutdown_connections 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9a9c06c8f0 0x7f9a9c06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ac015d9c0 0x7f9ac01f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ac015a7e0 0x7f9ac0156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 --2- 172.21.15.43:0/1065783250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ac00b5410 0x7f9ac014c180 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:00.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 >> 172.21.15.43:0/1065783250 conn(0x7f9ac0068090 msgr2=0x7f9ac0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.816+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 shutdown_connections 2024-04-26T20:27:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:00.817+0000 7f9ac4e00640 1 -- 172.21.15.43:0/1065783250 wait complete. 2024-04-26T20:27:00.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:01.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.044+0000 7f52abe00640 1 Processor -- start 2024-04-26T20:27:01.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.044+0000 7f52abe00640 1 -- start start 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.046+0000 7f52abe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 0x7f52ac0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.046+0000 7f52abe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52ac072e20 0x7f52ac0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.046+0000 7f52abe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52ac0737f0 con 0x7f52ac15b1f0 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.046+0000 7f52abe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f52ac0b4270 con 0x7f52ac072e20 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.046+0000 7f52abe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac073a70 con 0x7f52ac0b5690 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aaa00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f52ac15b1f0 0x7f52ac0b4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41142/0 (socket says 172.21.15.43:41142) 2024-04-26T20:27:01.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aaa00640 1 -- 172.21.15.43:0/2102616496 learned_addr learned my addr 172.21.15.43:0/2102616496 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:01.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52a2000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 0x7f52ac0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52ab400640 1 --2- 172.21.15.43:0/2102616496 >> v2:172.21.15.164:3300/0 conn(0x7f52ac072e20 0x7f52ac0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275849895 0 0) 0x7f529c001110 con 0x7f52ac15b1f0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5290005730 con 0x7f52ac15b1f0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 781366555 0 0) 0x7f5290005730 con 0x7f52ac15b1f0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.047+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 msgr2=0x7f52ac0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 0x7f52ac0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 >> v2:172.21.15.164:3300/0 conn(0x7f52ac072e20 msgr2=0x7f52ac0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 --2- 172.21.15.43:0/2102616496 >> v2:172.21.15.164:3300/0 conn(0x7f52ac072e20 0x7f52ac0732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac0b3940 con 0x7f52ac15b1f0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f529c001840 con 0x7f52ac15b1f0 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 >> v1:172.21.15.43:6789/0 conn(0x7f52ac15b1f0 legacy=0x7f52ac0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5290005bc0 0x7f5290005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f5290006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 0x7f52900073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac0b4270 con 0x7f5290006a10 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac073a70 con 0x7f5290005bc0 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f529000dbd0 con 0x7f52ac0b5690 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52a2000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f5290006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52aaa00640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5290005bc0 0x7f5290005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52a2000640 1 -- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 msgr2=0x7f52900073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52a2000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0b5690 0x7f52900073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52a2000640 1 -- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5290005bc0 msgr2=0x7f5290005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.048+0000 7f52a2000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5290005bc0 0x7f5290005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.049+0000 7f52a2000640 1 -- 172.21.15.43:0/2102616496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5290005730 con 0x7f5290006a10 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.050+0000 7f52a2000640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f5290006e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5294009ac0 tx=0x7f5290007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19924d6e41ea4953 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.050+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5294036070 con 0x7f5290006a10 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.050+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5294031400 con 0x7f5290006a10 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.050+0000 7f52aa000640 1 -- 172.21.15.43:0/2102616496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5294009690 con 0x7f5290006a10 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 -- 172.21.15.43:0/2102616496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 msgr2=0x7f5290006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 --2- 172.21.15.43:0/2102616496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f5290006e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5294009ac0 tx=0x7f5290007a40 comp rx=0 tx=0).stop 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52ab400640 1 -- 172.21.15.43:0/2102616496 reap_dead start 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 -- 172.21.15.43:0/2102616496 shutdown_connections 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 -- 172.21.15.43:0/2102616496 >> 172.21.15.43:0/2102616496 conn(0x7f52ac068090 msgr2=0x7f52ac068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 -- 172.21.15.43:0/2102616496 shutdown_connections 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.051+0000 7f52abe00640 1 -- 172.21.15.43:0/2102616496 wait complete. 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.053+0000 7f52abe00640 1 Processor -- start 2024-04-26T20:27:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.053+0000 7f52abe00640 1 -- start start 2024-04-26T20:27:01.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 0x7f52ac071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 0x7f52ac1f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac15b5d0 con 0x7f5290006a10 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac0b3f00 con 0x7f52ac0720b0 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52abe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac0b4700 con 0x7f52ac0afbd0 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57270/0 (socket says 172.21.15.43:57270) 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.054+0000 7f52aaa00640 1 -- 172.21.15.43:0/1386964831 learned_addr learned my addr 172.21.15.43:0/1386964831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 -- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 msgr2=0x7f52ac071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52a2000640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 0x7f52ac071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52ab400640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 0x7f52ac1f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 0x7f52ac071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 -- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 msgr2=0x7f52ac1f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 0x7f52ac1f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 -- 172.21.15.43:0/1386964831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac1f8e90 con 0x7f5290006a10 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52a2000640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 0x7f52ac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52ab400640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 0x7f52ac1f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52aaa00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e820 tx=0x7f529c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00c870 con 0x7f5290006a10 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529c018070 con 0x7f5290006a10 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c00cc50 con 0x7f5290006a10 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1f8bb0 con 0x7f5290006a10 2024-04-26T20:27:01.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.055+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac0af3d0 con 0x7f5290006a10 2024-04-26T20:27:01.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.057+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac069350 con 0x7f5290006a10 2024-04-26T20:27:01.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.058+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f529c013420 con 0x7f5290006a10 2024-04-26T20:27:01.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.059+0000 7f52a3400640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 0x7f528806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.059+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f529c08bc60 con 0x7f5290006a10 2024-04-26T20:27:01.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.059+0000 7f52a2000640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 0x7f528806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.060+0000 7f52a2000640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 0x7f528806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5294033000 tx=0x7f5294008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.064+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f52ac0737f0 con 0x7f5290006a10 2024-04-26T20:27:01.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.271+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52ac072490 con 0x7f5290006a10 2024-04-26T20:27:01.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.273+0000 7f52a3400640 1 -- 172.21.15.43:0/1386964831 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f529c059c50 con 0x7f5290006a10 2024-04-26T20:27:01.274 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:01.274 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 msgr2=0x7f528806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 0x7f528806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5294033000 tx=0x7f5294008030 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 msgr2=0x7f52ac14f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e820 tx=0x7f529c00ece0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 shutdown_connections 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f528806c920 0x7f528806ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52ac0afbd0 0x7f52ac071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52ac0720b0 0x7f52ac1f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 --2- 172.21.15.43:0/1386964831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5290006a10 0x7f52ac14f3f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 >> 172.21.15.43:0/1386964831 conn(0x7f52ac068090 msgr2=0x7f52ac0b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:01.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 shutdown_connections 2024-04-26T20:27:01.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.276+0000 7f52abe00640 1 -- 172.21.15.43:0/1386964831 wait complete. 2024-04-26T20:27:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:01.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:01.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.511+0000 7fb087000640 1 Processor -- start 2024-04-26T20:27:01.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.512+0000 7fb087000640 1 -- start start 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.513+0000 7fb087000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb0800d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.513+0000 7fb087000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb0800b3cb0 0x7fb0800d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.513+0000 7fb087000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0800d2d70 con 0x7fb0800d3660 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.513+0000 7fb087000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb08015f560 con 0x7fb0800b3cb0 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.513+0000 7fb087000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0800b2750 con 0x7fb08015f180 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb086600640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb0800d3660 0x7fb0800d1570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41152/0 (socket says 172.21.15.43:41152) 2024-04-26T20:27:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb086600640 1 -- 172.21.15.43:0/2258312431 learned_addr learned my addr 172.21.15.43:0/2258312431 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:01.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb085c00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb0800d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb085200640 1 --2- 172.21.15.43:0/2258312431 >> v2:172.21.15.164:3300/0 conn(0x7fb0800b3cb0 0x7fb0800d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511682339 0 0) 0x7fb0800d2d70 con 0x7fb0800d3660 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.514+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb064003520 con 0x7fb0800d3660 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1970726565 0 0) 0x7fb064003520 con 0x7fb0800d3660 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 msgr2=0x7fb0800d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb0800d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 >> v2:172.21.15.164:3300/0 conn(0x7fb0800b3cb0 msgr2=0x7fb0800d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2258312431 >> v2:172.21.15.164:3300/0 conn(0x7fb0800b3cb0 0x7fb0800d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0800d06e0 con 0x7fb0800d3660 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fb078001670 con 0x7fb0800d3660 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 >> v1:172.21.15.43:6789/0 conn(0x7fb0800d3660 legacy=0x7fb0800d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb064003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003fa0 0x7fb064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb08015f560 con 0x7fb064004d40 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800b2750 con 0x7fb064003fa0 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb06400b9f0 con 0x7fb08015f180 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb086600640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.515+0000 7fb085c00640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb064003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb085200640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003fa0 0x7fb064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb086600640 1 -- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 msgr2=0x7fb064003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb086600640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb08015f180 0x7fb064003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb086600640 1 -- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003fa0 msgr2=0x7fb064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb086600640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003fa0 0x7fb064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb086600640 1 -- 172.21.15.43:0/2258312431 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb064003520 con 0x7fb064004d40 2024-04-26T20:27:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.516+0000 7fb085200640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb064003fa0 0x7fb064004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb086600640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c210 tx=0x7fb064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce5fe41eb99a126 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078007d50 con 0x7fb064004d40 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb078005af0 con 0x7fb064004d40 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb07fe00640 1 -- 172.21.15.43:0/2258312431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078007070 con 0x7fb064004d40 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb087000640 1 -- 172.21.15.43:0/2258312431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 msgr2=0x7fb0640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.517+0000 7fb087000640 1 --2- 172.21.15.43:0/2258312431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064004d40 0x7fb0640051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c210 tx=0x7fb064005830 comp rx=0 tx=0).stop 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.518+0000 7fb086600640 1 -- 172.21.15.43:0/2258312431 reap_dead start 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.518+0000 7fb087000640 1 -- 172.21.15.43:0/2258312431 shutdown_connections 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.518+0000 7fb087000640 1 -- 172.21.15.43:0/2258312431 >> 172.21.15.43:0/2258312431 conn(0x7fb0800d4bd0 msgr2=0x7fb0800d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.518+0000 7fb087000640 1 -- 172.21.15.43:0/2258312431 shutdown_connections 2024-04-26T20:27:01.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.518+0000 7fb087000640 1 -- 172.21.15.43:0/2258312431 wait complete. 2024-04-26T20:27:01.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.520+0000 7fb087000640 1 Processor -- start 2024-04-26T20:27:01.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.520+0000 7fb087000640 1 -- start start 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 0x7fb080156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 0x7fb0800b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800d2a40 con 0x7fb0800ba1e0 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0800d0ca0 con 0x7fb080154f40 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb087000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0800d11c0 con 0x7fb064004d40 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb086600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb086600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57298/0 (socket says 172.21.15.43:57298) 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb086600640 1 -- 172.21.15.43:0/1045288795 learned_addr learned my addr 172.21.15.43:0/1045288795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb085200640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 0x7fb0800b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.521+0000 7fb085c00640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 0x7fb080156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 -- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 msgr2=0x7fb080156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 0x7fb080156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 -- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 msgr2=0x7fb0800b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 0x7fb0800b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 -- 172.21.15.43:0/1045288795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0800ba9d0 con 0x7fb0800ba1e0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb085c00640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 0x7fb080156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb085200640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 0x7fb0800b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb086600640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb078034040 tx=0x7fb078001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078007d50 con 0x7fb0800ba1e0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb078044070 con 0x7fb0800ba1e0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0800ba6f0 con 0x7fb0800ba1e0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.522+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078001c90 con 0x7fb0800ba1e0 2024-04-26T20:27:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.523+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb080155d90 con 0x7fb0800ba1e0 2024-04-26T20:27:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.524+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0800506c0 con 0x7fb0800ba1e0 2024-04-26T20:27:01.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.525+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb07803e480 con 0x7fb0800ba1e0 2024-04-26T20:27:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.525+0000 7fb07d600640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 0x7fb06006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.526+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fb0780ac830 con 0x7fb0800ba1e0 2024-04-26T20:27:01.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.526+0000 7fb085c00640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 0x7fb06006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.527+0000 7fb085c00640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 0x7fb06006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb070007440 tx=0x7fb07000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.530+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb07807a830 con 0x7fb0800ba1e0 2024-04-26T20:27:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.733+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb080156ad0 con 0x7fb0800ba1e0 2024-04-26T20:27:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.734+0000 7fb07d600640 1 -- 172.21.15.43:0/1045288795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb07807a130 con 0x7fb0800ba1e0 2024-04-26T20:27:01.734 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.737+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 msgr2=0x7fb06006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.737+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 0x7fb06006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb070007440 tx=0x7fb07000b040 comp rx=0 tx=0).stop 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.737+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 msgr2=0x7fb0800b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.737+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb078034040 tx=0x7fb078001a40 comp rx=0 tx=0).stop 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 shutdown_connections 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb06006c8f0 0x7fb06006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0800ba1e0 0x7fb0800b28d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb080154f40 0x7fb0800b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 --2- 172.21.15.43:0/1045288795 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb064004d40 0x7fb080156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 >> 172.21.15.43:0/1045288795 conn(0x7fb0800d4bd0 msgr2=0x7fb0800d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 shutdown_connections 2024-04-26T20:27:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.738+0000 7fb087000640 1 -- 172.21.15.43:0/1045288795 wait complete. 2024-04-26T20:27:01.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:01.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.960+0000 7f21cd400640 1 Processor -- start 2024-04-26T20:27:01.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.960+0000 7f21cd400640 1 -- start start 2024-04-26T20:27:01.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cd400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 0x7f21c80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cd400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21c815d9c0 0x7f21c80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cd400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c80b3900 con 0x7f21c80b5410 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cd400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f21c815abc0 con 0x7f21c815d9c0 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cd400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80b3b70 con 0x7f21c815a7e0 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cca00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21c815d9c0 0x7f21c80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cca00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21c815d9c0 0x7f21c80b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57310/0 (socket says 172.21.15.43:57310) 2024-04-26T20:27:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.962+0000 7f21cca00640 1 -- 172.21.15.43:0/1479439643 learned_addr learned my addr 172.21.15.43:0/1479439643 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21cca00640 1 -- 172.21.15.43:0/1479439643 >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 msgr2=0x7f21c80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21c7e00640 1 --2- 172.21.15.43:0/1479439643 >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 0x7f21c80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21cca00640 1 --2- 172.21.15.43:0/1479439643 >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 0x7f21c80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21cca00640 1 -- 172.21.15.43:0/1479439643 >> v1:172.21.15.164:6789/0 conn(0x7f21c80b5410 legacy=0x7f21c815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21cca00640 1 -- 172.21.15.43:0/1479439643 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80b3e50 con 0x7f21c815d9c0 2024-04-26T20:27:01.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.963+0000 7f21c7e00640 1 --2- 172.21.15.43:0/1479439643 >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 0x7f21c80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.964+0000 7f21cca00640 1 --2- 172.21.15.43:0/1479439643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c815d9c0 0x7f21c80b5890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f21c80d1620 tx=0x7f21c002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60021c3ed547aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.964+0000 7f21c7400640 1 -- 172.21.15.43:0/1479439643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0036070 con 0x7f21c815d9c0 2024-04-26T20:27:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.964+0000 7f21c7400640 1 -- 172.21.15.43:0/1479439643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c0031600 con 0x7f21c815d9c0 2024-04-26T20:27:01.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.964+0000 7f21c7400640 1 -- 172.21.15.43:0/1479439643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c00319d0 con 0x7f21c815d9c0 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 -- 172.21.15.43:0/1479439643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c815d9c0 msgr2=0x7f21c80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 --2- 172.21.15.43:0/1479439643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c815d9c0 0x7f21c80b5890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f21c80d1620 tx=0x7f21c002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 -- 172.21.15.43:0/1479439643 shutdown_connections 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 --2- 172.21.15.43:0/1479439643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c815d9c0 0x7f21c80b5890 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 --2- 172.21.15.43:0/1479439643 >> v2:172.21.15.164:3300/0 conn(0x7f21c815a7e0 0x7f21c80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 -- 172.21.15.43:0/1479439643 >> 172.21.15.43:0/1479439643 conn(0x7f21c8068090 msgr2=0x7f21c80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 -- 172.21.15.43:0/1479439643 shutdown_connections 2024-04-26T20:27:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.965+0000 7f21cd400640 1 -- 172.21.15.43:0/1479439643 wait complete. 2024-04-26T20:27:01.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.967+0000 7f21cd400640 1 Processor -- start 2024-04-26T20:27:01.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.967+0000 7f21cd400640 1 -- start start 2024-04-26T20:27:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c815a7e0 0x7f21c80b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c815d9c0 0x7f21c815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80b4b90 con 0x7f21c80b5410 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21c815e010 con 0x7f21c815a7e0 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21cd400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f21c815e2e0 con 0x7f21c815d9c0 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.968+0000 7f21c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57316/0 (socket says 172.21.15.43:57316) 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 -- 172.21.15.43:0/1204881591 learned_addr learned my addr 172.21.15.43:0/1204881591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21bfe00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c815a7e0 0x7f21c80b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21cca00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c815d9c0 0x7f21c815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 -- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c815d9c0 msgr2=0x7f21c815ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c815d9c0 0x7f21c815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 -- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c815a7e0 msgr2=0x7f21c80b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:01.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c815a7e0 0x7f21c80b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 -- 172.21.15.43:0/1204881591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c8146790 con 0x7f21c80b5410 2024-04-26T20:27:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.969+0000 7f21c7e00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c7f0 tx=0x7f21b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.970+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8015070 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.970+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b8007a30 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.970+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8007e00 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.970+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c80b0c90 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.970+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c80b1250 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.971+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c8068470 con 0x7f21c80b5410 2024-04-26T20:27:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.972+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21b8018070 con 0x7f21c80b5410 2024-04-26T20:27:01.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.973+0000 7f21c4c00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 0x7f21a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:01.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.973+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f21b808bf00 con 0x7f21c80b5410 2024-04-26T20:27:01.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.974+0000 7f21bfe00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 0x7f21a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:01.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.974+0000 7f21bfe00640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 0x7f21a406ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21c8149bc0 tx=0x7f21b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:01.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:01.978+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21c8068470 con 0x7f21c80b5410 2024-04-26T20:27:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.184+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f21c80506c0 con 0x7f21c80b5410 2024-04-26T20:27:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.185+0000 7f21c4c00640 1 -- 172.21.15.43:0/1204881591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f21c8146790 con 0x7f21c80b5410 2024-04-26T20:27:02.186 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 msgr2=0x7f21a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 0x7f21a406ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21c8149bc0 tx=0x7f21b0009040 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 msgr2=0x7f21c8146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f21b800c7f0 tx=0x7f21b800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 shutdown_connections 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f21a406c920 0x7f21a406ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21c815d9c0 0x7f21c815ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21c815a7e0 0x7f21c80b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 --2- 172.21.15.43:0/1204881591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c80b5410 0x7f21c8146250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 >> 172.21.15.43:0/1204881591 conn(0x7f21c8068090 msgr2=0x7f21c815b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 shutdown_connections 2024-04-26T20:27:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.188+0000 7f21cd400640 1 -- 172.21.15.43:0/1204881591 wait complete. 2024-04-26T20:27:02.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:02.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.422+0000 7f520b200640 1 Processor -- start 2024-04-26T20:27:02.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.422+0000 7f520b200640 1 -- start start 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f520b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f520c0b5200 0x7f520c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f520b200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c0b57c0 con 0x7f520c156100 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f520b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c15c230 con 0x7f520c0d12e0 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f520b200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f520c15c510 con 0x7f520c0b5200 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f5209e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f520c156100 0x7f520c0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41168/0 (socket says 172.21.15.43:41168) 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f5209e00640 1 -- 172.21.15.43:0/3193618630 learned_addr learned my addr 172.21.15.43:0/3193618630 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.424+0000 7f520a800640 1 --2- 172.21.15.43:0/3193618630 >> v2:172.21.15.43:3300/0 conn(0x7f520c0b5200 0x7f520c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.425+0000 7f5208a00640 1 -- 172.21.15.43:0/3193618630 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139150471 0 0) 0x7f51fc001110 con 0x7f520c156100 2024-04-26T20:27:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.425+0000 7f5208a00640 1 -- 172.21.15.43:0/3193618630 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51f0003520 con 0x7f520c156100 2024-04-26T20:27:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.425+0000 7f520a800640 1 -- 172.21.15.43:0/3193618630 >> v1:172.21.15.164:6789/0 conn(0x7f520c0d12e0 legacy=0x7f520c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.425+0000 7f520a800640 1 -- 172.21.15.43:0/3193618630 >> v1:172.21.15.43:6789/0 conn(0x7f520c156100 legacy=0x7f520c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.425+0000 7f520a800640 1 -- 172.21.15.43:0/3193618630 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c15c7f0 con 0x7f520c0b5200 2024-04-26T20:27:02.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.426+0000 7f520a800640 1 --2- 172.21.15.43:0/3193618630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0b3c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5204009d90 tx=0x7f520402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f7d478f3601183 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.426+0000 7f5208a00640 1 -- 172.21.15.43:0/3193618630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204007740 con 0x7f520c0b5200 2024-04-26T20:27:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.426+0000 7f5208a00640 1 -- 172.21.15.43:0/3193618630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52040078e0 con 0x7f520c0b5200 2024-04-26T20:27:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.426+0000 7f5208a00640 1 -- 172.21.15.43:0/3193618630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204007cb0 con 0x7f520c0b5200 2024-04-26T20:27:02.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 -- 172.21.15.43:0/3193618630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 msgr2=0x7f520c0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 --2- 172.21.15.43:0/3193618630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0b3c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5204009d90 tx=0x7f520402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 -- 172.21.15.43:0/3193618630 shutdown_connections 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 --2- 172.21.15.43:0/3193618630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0b3c90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 -- 172.21.15.43:0/3193618630 >> 172.21.15.43:0/3193618630 conn(0x7f520c0684e0 msgr2=0x7f520c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 -- 172.21.15.43:0/3193618630 shutdown_connections 2024-04-26T20:27:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.427+0000 7f520b200640 1 -- 172.21.15.43:0/3193618630 wait complete. 2024-04-26T20:27:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.429+0000 7f520b200640 1 Processor -- start 2024-04-26T20:27:02.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.429+0000 7f520b200640 1 -- start start 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520c0d12e0 0x7f520c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f520c156100 0x7f520c1f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f520c0b39e0 con 0x7f520c0b5200 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f520c0b3860 con 0x7f520c0d12e0 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f520b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f520c0b41d0 con 0x7f520c156100 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f5209e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f5209e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57336/0 (socket says 172.21.15.43:57336) 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.430+0000 7f5209e00640 1 -- 172.21.15.43:0/811748745 learned_addr learned my addr 172.21.15.43:0/811748745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209400640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520c0d12e0 0x7f520c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f520a800640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f520c156100 0x7f520c1f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 -- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f520c156100 msgr2=0x7f520c1f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f520c156100 0x7f520c1f91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 -- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520c0d12e0 msgr2=0x7f520c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520c0d12e0 0x7f520c0cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 -- 172.21.15.43:0/811748745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c1f5ce0 con 0x7f520c0b5200 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5209e00640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00c430 tx=0x7f51fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc017070 con 0x7f520c0b5200 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc005ce0 con 0x7f520c0b5200 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c1f5fc0 con 0x7f520c0b5200 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0072d0 con 0x7f520c0b5200 2024-04-26T20:27:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.431+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1f8cb0 con 0x7f520c0b5200 2024-04-26T20:27:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.433+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c1f5a60 con 0x7f520c0b5200 2024-04-26T20:27:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.434+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51fc01a070 con 0x7f520c0b5200 2024-04-26T20:27:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.434+0000 7f5202000640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 0x7f51ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.435+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f51fc08b240 con 0x7f520c0b5200 2024-04-26T20:27:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.435+0000 7f5209400640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 0x7f51ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.436+0000 7f5209400640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 0x7f51ec06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c15fd70 tx=0x7f51f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.440+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f520c1f5a60 con 0x7f520c0b5200 2024-04-26T20:27:02.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.644+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f520c0506c0 con 0x7f520c0b5200 2024-04-26T20:27:02.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.645+0000 7f5202000640 1 -- 172.21.15.43:0/811748745 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f520c0b57c0 con 0x7f520c0b5200 2024-04-26T20:27:02.646 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:02.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 msgr2=0x7f51ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 0x7f51ec06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f520c15fd70 tx=0x7f51f8009040 comp rx=0 tx=0).stop 2024-04-26T20:27:02.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 msgr2=0x7f520c0af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00c430 tx=0x7f51fc00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 shutdown_connections 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f51ec06c8f0 0x7f51ec06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f520c156100 0x7f520c1f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f520c0d12e0 0x7f520c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 --2- 172.21.15.43:0/811748745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f520c0b5200 0x7f520c0af2d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 >> 172.21.15.43:0/811748745 conn(0x7f520c0684e0 msgr2=0x7f520c06dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 shutdown_connections 2024-04-26T20:27:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.648+0000 7f520b200640 1 -- 172.21.15.43:0/811748745 wait complete. 2024-04-26T20:27:02.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.881+0000 7f1103200640 1 Processor -- start 2024-04-26T20:27:02.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.881+0000 7f1103200640 1 -- start start 2024-04-26T20:27:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1103200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10fc15f180 0x7f10fc0d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1103200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f10fc0d3830 0x7f10fc0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1103200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc0d2d70 con 0x7f10fc0b3cb0 2024-04-26T20:27:02.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1103200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc15f560 con 0x7f10fc15f180 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1103200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc0d06e0 con 0x7f10fc0d3830 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.883+0000 7f1101e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10fc15f180 0x7f10fc0d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10fc15f180 0x7f10fc0d56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57342/0 (socket says 172.21.15.43:57342) 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 -- 172.21.15.43:0/862465651 learned_addr learned my addr 172.21.15.43:0/862465651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1102800640 1 --2- 172.21.15.43:0/862465651 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0d3830 0x7f10fc0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 -- 172.21.15.43:0/862465651 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0d3830 msgr2=0x7f10fc0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 --2- 172.21.15.43:0/862465651 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0d3830 0x7f10fc0b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 -- 172.21.15.43:0/862465651 >> v1:172.21.15.43:6789/0 conn(0x7f10fc0b3cb0 legacy=0x7f10fc0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.884+0000 7f1101e00640 1 -- 172.21.15.43:0/862465651 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0d09c0 con 0x7f10fc15f180 2024-04-26T20:27:02.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.886+0000 7f1101e00640 1 --2- 172.21.15.43:0/862465651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc15f180 0x7f10fc0d56f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f10fc08d540 tx=0x7f10f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cb1362219f6b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.886+0000 7f1100a00640 1 -- 172.21.15.43:0/862465651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0037070 con 0x7f10fc15f180 2024-04-26T20:27:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.886+0000 7f1100a00640 1 -- 172.21.15.43:0/862465651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f0032440 con 0x7f10fc15f180 2024-04-26T20:27:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.886+0000 7f1100a00640 1 -- 172.21.15.43:0/862465651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0032810 con 0x7f10fc15f180 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 -- 172.21.15.43:0/862465651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc15f180 msgr2=0x7f10fc0d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 --2- 172.21.15.43:0/862465651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc15f180 0x7f10fc0d56f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f10fc08d540 tx=0x7f10f002a270 comp rx=0 tx=0).stop 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 -- 172.21.15.43:0/862465651 shutdown_connections 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 --2- 172.21.15.43:0/862465651 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0d3830 0x7f10fc0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 --2- 172.21.15.43:0/862465651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc15f180 0x7f10fc0d56f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 -- 172.21.15.43:0/862465651 >> 172.21.15.43:0/862465651 conn(0x7f10fc0d4bd0 msgr2=0x7f10fc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 -- 172.21.15.43:0/862465651 shutdown_connections 2024-04-26T20:27:02.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.887+0000 7f1103200640 1 -- 172.21.15.43:0/862465651 wait complete. 2024-04-26T20:27:02.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.889+0000 7f1103200640 1 Processor -- start 2024-04-26T20:27:02.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.889+0000 7f1103200640 1 -- start start 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b3cb0 0x7f10fc14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc15f180 0x7f10fc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0d11c0 con 0x7f10fc0d3830 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0d2a40 con 0x7f10fc15f180 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1103200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc156450 con 0x7f10fc0b3cb0 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1101400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1101400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57348/0 (socket says 172.21.15.43:57348) 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.890+0000 7f1101400640 1 -- 172.21.15.43:0/3280344776 learned_addr learned my addr 172.21.15.43:0/3280344776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:02.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101e00640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b3cb0 0x7f10fc14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1102800640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc15f180 0x7f10fc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 -- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b3cb0 msgr2=0x7f10fc14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b3cb0 0x7f10fc14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 -- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc15f180 msgr2=0x7f10fc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc15f180 0x7f10fc15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 -- 172.21.15.43:0/3280344776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc14d0a0 con 0x7f10fc0d3830 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1101400640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3f0 tx=0x7f10ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec017070 con 0x7f10fc0d3830 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ec0052f0 con 0x7f10fc0d3830 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003850 con 0x7f10fc0d3830 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.891+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc0ba5f0 con 0x7f10fc0d3830 2024-04-26T20:27:02.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.892+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc0ca890 con 0x7f10fc0d3830 2024-04-26T20:27:02.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.893+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc0506c0 con 0x7f10fc0d3830 2024-04-26T20:27:02.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.894+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10ec01a070 con 0x7f10fc0d3830 2024-04-26T20:27:02.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.895+0000 7f10f6000640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:02.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.895+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f10ec08b380 con 0x7f10fc0d3830 2024-04-26T20:27:02.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.895+0000 7f1101e00640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:02.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.896+0000 7f1101e00640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 0x7f10e006ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:02.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:02.899+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10ec0592e0 con 0x7f10fc0d3830 2024-04-26T20:27:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.100+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f10fc0d54a0 con 0x7f10fc0d3830 2024-04-26T20:27:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.101+0000 7f10f6000640 1 -- 172.21.15.43:0/3280344776 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f10ec058be0 con 0x7f10fc0d3830 2024-04-26T20:27:03.101 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 msgr2=0x7f10e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 0x7f10e006ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10f0034000 tx=0x7f10f002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 msgr2=0x7f10fc0ba0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c3f0 tx=0x7f10ec00c8b0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 shutdown_connections 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10e006c920 0x7f10e006ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc15f180 0x7f10fc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10fc0d3830 0x7f10fc0ba0b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 --2- 172.21.15.43:0/3280344776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b3cb0 0x7f10fc14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 >> 172.21.15.43:0/3280344776 conn(0x7f10fc0d4bd0 msgr2=0x7f10fc15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 shutdown_connections 2024-04-26T20:27:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.104+0000 7f1103200640 1 -- 172.21.15.43:0/3280344776 wait complete. 2024-04-26T20:27:03.135 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-04-26T20:27:03.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:03.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.350+0000 7f1101c00640 1 Processor -- start 2024-04-26T20:27:03.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.350+0000 7f1101c00640 1 -- start start 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f1101c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f10fc0cfb20 0x7f10fc0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f1101c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc0d2160 con 0x7f10fc0b8570 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f1101c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc0cd280 con 0x7f10fc0d36b0 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f1101c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc0cf050 con 0x7f10fc0cfb20 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f10faa00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10fc0b8570 0x7f10fc0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41184/0 (socket says 172.21.15.43:41184) 2024-04-26T20:27:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.352+0000 7f10faa00640 1 -- 172.21.15.43:0/1591023542 learned_addr learned my addr 172.21.15.43:0/1591023542 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92999630 0 0) 0x7f10fc0d2160 con 0x7f10fc0b8570 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10e0003520 con 0x7f10fc0b8570 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10f3e00640 1 --2- 172.21.15.43:0/1591023542 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0cfb20 0x7f10fc0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674824308 0 0) 0x7f10e0003520 con 0x7f10fc0b8570 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0cfb20 msgr2=0x7f10fc0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 --2- 172.21.15.43:0/1591023542 >> v2:172.21.15.164:3300/0 conn(0x7f10fc0cfb20 0x7f10fc0d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 >> v1:172.21.15.164:6789/0 conn(0x7f10fc0d36b0 legacy=0x7f10fc15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0cd560 con 0x7f10fc0b8570 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.353+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f10ec001670 con 0x7f10fc0b8570 2024-04-26T20:27:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 >> v1:172.21.15.43:6789/0 conn(0x7f10fc0b8570 legacy=0x7f10fc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10e00039b0 0x7f10e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10e00051c0 0x7f10e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0cf050 con 0x7f10e0004800 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10e000c060 con 0x7f10e00051c0 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f10e000c260 con 0x7f10e00039b0 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10faa00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10e00039b0 0x7f10e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 -- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10e00039b0 msgr2=0x7f10e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10e00039b0 0x7f10e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 -- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10e00051c0 msgr2=0x7f10e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10e00051c0 0x7f10e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10f3e00640 1 -- 172.21.15.43:0/1591023542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e0003520 con 0x7f10e0004800 2024-04-26T20:27:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10fb400640 1 -- 172.21.15.43:0/1591023542 reap_dead start 2024-04-26T20:27:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.354+0000 7f10faa00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10e00039b0 0x7f10e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f10f3e00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10e0004c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a90 tx=0x7f10e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817d6a8b9bc87f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4036070 con 0x7f10e0004800 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e4031400 con 0x7f10e0004800 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f10fa000640 1 -- 172.21.15.43:0/1591023542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4009690 con 0x7f10e0004800 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f1101c00640 1 -- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 msgr2=0x7f10e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.356+0000 7f1101c00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10e0004c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009a90 tx=0x7f10e0005830 comp rx=0 tx=0).stop 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.357+0000 7f1101c00640 1 -- 172.21.15.43:0/1591023542 shutdown_connections 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.357+0000 7f1101c00640 1 --2- 172.21.15.43:0/1591023542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10e0004c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.357+0000 7f1101c00640 1 -- 172.21.15.43:0/1591023542 >> 172.21.15.43:0/1591023542 conn(0x7f10fc0d2b90 msgr2=0x7f10fc0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.357+0000 7f1101c00640 1 -- 172.21.15.43:0/1591023542 shutdown_connections 2024-04-26T20:27:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.357+0000 7f1101c00640 1 -- 172.21.15.43:0/1591023542 wait complete. 2024-04-26T20:27:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.359+0000 7f1101c00640 1 Processor -- start 2024-04-26T20:27:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.359+0000 7f1101c00640 1 -- start start 2024-04-26T20:27:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc0cc070 0x7f10fc1f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b66f0 0x7f10fc0b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0d1df0 con 0x7f10e0004800 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0cdb20 con 0x7f10fc0cc070 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f1101c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc0b59e0 con 0x7f10fc0b66f0 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57362/0 (socket says 172.21.15.43:57362) 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 -- 172.21.15.43:0/3733858054 learned_addr learned my addr 172.21.15.43:0/3733858054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 -- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b66f0 msgr2=0x7f10fc0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b66f0 0x7f10fc0b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 -- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc0cc070 msgr2=0x7f10fc1f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc0cc070 0x7f10fc1f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.360+0000 7f10faa00640 1 -- 172.21.15.43:0/3733858054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc1f53f0 con 0x7f10e0004800 2024-04-26T20:27:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f10faa00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00d810 tx=0x7f10ec00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec019070 con 0x7f10e0004800 2024-04-26T20:27:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ec0074e0 con 0x7f10e0004800 2024-04-26T20:27:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004a00 con 0x7f10e0004800 2024-04-26T20:27:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc1f56d0 con 0x7f10e0004800 2024-04-26T20:27:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.361+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc0cc4f0 con 0x7f10e0004800 2024-04-26T20:27:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.362+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc1f5bf0 con 0x7f10e0004800 2024-04-26T20:27:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.363+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10ec01c070 con 0x7f10e0004800 2024-04-26T20:27:03.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.364+0000 7f10f2a00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 0x7f10d806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.364+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f10ec08b9a0 con 0x7f10e0004800 2024-04-26T20:27:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.364+0000 7f10f3e00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 0x7f10d806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.365+0000 7f10f3e00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 0x7f10d806ee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10e4033000 tx=0x7f10e402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.369+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10ec059a10 con 0x7f10e0004800 2024-04-26T20:27:03.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.577+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10fc069350 con 0x7f10e0004800 2024-04-26T20:27:03.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.580+0000 7f10f2a00640 1 -- 172.21.15.43:0/3733858054 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f10ec059310 con 0x7f10e0004800 2024-04-26T20:27:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:03.581 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.582+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 msgr2=0x7f10d806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.582+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 0x7f10d806ee00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10e4033000 tx=0x7f10e402e000 comp rx=0 tx=0).stop 2024-04-26T20:27:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.582+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 msgr2=0x7f10fc0a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.582+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00d810 tx=0x7f10ec00dcd0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 shutdown_connections 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f10d806c940 0x7f10d806ee00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10fc0b66f0 0x7f10fc0b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10fc0cc070 0x7f10fc1f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 --2- 172.21.15.43:0/3733858054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10e0004800 0x7f10fc0a04b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 >> 172.21.15.43:0/3733858054 conn(0x7f10fc0d2b90 msgr2=0x7f10fc071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 shutdown_connections 2024-04-26T20:27:03.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.583+0000 7f1101c00640 1 -- 172.21.15.43:0/3733858054 wait complete. 2024-04-26T20:27:03.602 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:03.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:03.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.824+0000 7ff397e00640 1 Processor -- start 2024-04-26T20:27:03.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.825+0000 7ff397e00640 1 -- start start 2024-04-26T20:27:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.826+0000 7ff397e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3980d3830 0x7ff3980d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.826+0000 7ff397e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3980d2d70 con 0x7ff3980b3cb0 2024-04-26T20:27:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.826+0000 7ff397e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3980d06e0 con 0x7ff39815f180 2024-04-26T20:27:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.826+0000 7ff397e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff39815f560 con 0x7ff3980d3830 2024-04-26T20:27:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff397400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff3980b3cb0 0x7ff3980b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41200/0 (socket says 172.21.15.43:41200) 2024-04-26T20:27:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff397400640 1 -- 172.21.15.43:0/3992370902 learned_addr learned my addr 172.21.15.43:0/3992370902 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff396000640 1 --2- 172.21.15.43:0/3992370902 >> v2:172.21.15.43:3300/0 conn(0x7ff3980d3830 0x7ff3980d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff395600640 1 -- 172.21.15.43:0/3992370902 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1100755894 0 0) 0x7ff3980d2d70 con 0x7ff3980b3cb0 2024-04-26T20:27:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff395600640 1 -- 172.21.15.43:0/3992370902 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c003520 con 0x7ff3980b3cb0 2024-04-26T20:27:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff396000640 1 -- 172.21.15.43:0/3992370902 >> v1:172.21.15.164:6789/0 conn(0x7ff39815f180 legacy=0x7ff3980d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff396000640 1 -- 172.21.15.43:0/3992370902 >> v1:172.21.15.43:6789/0 conn(0x7ff3980b3cb0 legacy=0x7ff3980b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.827+0000 7ff396000640 1 -- 172.21.15.43:0/3992370902 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3980d09c0 con 0x7ff3980d3830 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff396000640 1 --2- 172.21.15.43:0/3992370902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980d3830 0x7ff3980d56f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097a0 tx=0x7ff38402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf92d1ccc20a79f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff395600640 1 -- 172.21.15.43:0/3992370902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384037070 con 0x7ff3980d3830 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff395600640 1 -- 172.21.15.43:0/3992370902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384032440 con 0x7ff3980d3830 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff395600640 1 -- 172.21.15.43:0/3992370902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840327f0 con 0x7ff3980d3830 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff397e00640 1 -- 172.21.15.43:0/3992370902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980d3830 msgr2=0x7ff3980d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff397e00640 1 --2- 172.21.15.43:0/3992370902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980d3830 0x7ff3980d56f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097a0 tx=0x7ff38402a230 comp rx=0 tx=0).stop 2024-04-26T20:27:03.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff397e00640 1 -- 172.21.15.43:0/3992370902 shutdown_connections 2024-04-26T20:27:03.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.829+0000 7ff397e00640 1 --2- 172.21.15.43:0/3992370902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980d3830 0x7ff3980d56f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.830+0000 7ff397e00640 1 -- 172.21.15.43:0/3992370902 >> 172.21.15.43:0/3992370902 conn(0x7ff3980d4bd0 msgr2=0x7ff3980d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:03.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.830+0000 7ff397e00640 1 -- 172.21.15.43:0/3992370902 shutdown_connections 2024-04-26T20:27:03.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.830+0000 7ff397e00640 1 -- 172.21.15.43:0/3992370902 wait complete. 2024-04-26T20:27:03.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.832+0000 7ff397e00640 1 Processor -- start 2024-04-26T20:27:03.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.832+0000 7ff397e00640 1 -- start start 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3980d3830 0x7ff3980b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff39815f180 0x7ff3980cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980d11c0 con 0x7ff3980b3cb0 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3980d2a40 con 0x7ff3980d3830 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff3980b4bc0 con 0x7ff39815f180 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff396a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff396a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57392/0 (socket says 172.21.15.43:57392) 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff396a00640 1 -- 172.21.15.43:0/845476332 learned_addr learned my addr 172.21.15.43:0/845476332 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff397400640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff39815f180 0x7ff3980cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.833+0000 7ff396a00640 1 -- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff39815f180 msgr2=0x7ff3980cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396a00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff39815f180 0x7ff3980cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396a00640 1 -- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3980d3830 msgr2=0x7ff3980b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396000640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3980d3830 0x7ff3980b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396a00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3980d3830 0x7ff3980b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396a00640 1 -- 172.21.15.43:0/845476332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3980b4ea0 con 0x7ff3980b3cb0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff396a00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c3c0 tx=0x7ff38800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3880052d0 con 0x7ff3980b3cb0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388005490 con 0x7ff3980b3cb0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3980b5180 con 0x7ff3980b3cb0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388007260 con 0x7ff3980b3cb0 2024-04-26T20:27:03.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.834+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3981541e0 con 0x7ff3980b3cb0 2024-04-26T20:27:03.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.835+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3980506c0 con 0x7ff3980b3cb0 2024-04-26T20:27:03.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.837+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff388007400 con 0x7ff3980b3cb0 2024-04-26T20:27:03.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.837+0000 7ff38ea00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:03.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.838+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7ff38808b480 con 0x7ff3980b3cb0 2024-04-26T20:27:03.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.838+0000 7ff396000640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.839+0000 7ff396000640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 0x7ff37806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:03.843+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff388059480 con 0x7ff3980b3cb0 2024-04-26T20:27:04.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.050+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff39815c220 con 0x7ff3980b3cb0 2024-04-26T20:27:04.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.052+0000 7ff38ea00640 1 -- 172.21.15.43:0/845476332 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff388058d80 con 0x7ff3980b3cb0 2024-04-26T20:27:04.052 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.054+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 msgr2=0x7ff37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.054+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 0x7ff37806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).stop 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.054+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 msgr2=0x7ff398153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.054+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c3c0 tx=0x7ff38800c880 comp rx=0 tx=0).stop 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 shutdown_connections 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff39815f180 0x7ff3980cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3980d3830 0x7ff3980b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 --2- 172.21.15.43:0/845476332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3980b3cb0 0x7ff398153ca0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 >> 172.21.15.43:0/845476332 conn(0x7ff3980d4bd0 msgr2=0x7ff39815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 shutdown_connections 2024-04-26T20:27:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.055+0000 7ff397e00640 1 -- 172.21.15.43:0/845476332 wait complete. 2024-04-26T20:27:04.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:04.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.295+0000 7fbbc5600640 1 Processor -- start 2024-04-26T20:27:04.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.295+0000 7fbbc5600640 1 -- start start 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.296+0000 7fbbc5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbbc00d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbc5600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc00b5160 con 0x7fbbc00d2d70 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbc5600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc00b5440 con 0x7fbbc00d3780 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbc5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc00b5d70 con 0x7fbbc00d31d0 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbbe000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbbc00d2d70 0x7fbbc00cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41210/0 (socket says 172.21.15.43:41210) 2024-04-26T20:27:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbbe000640 1 -- 172.21.15.43:0/2518463504 learned_addr learned my addr 172.21.15.43:0/2518463504 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3410524037 0 0) 0x7fbbc00b5160 con 0x7fbbc00d2d70 2024-04-26T20:27:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbba4003520 con 0x7fbbc00d2d70 2024-04-26T20:27:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.297+0000 7fbbbf400640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbbc00d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2950095955 0 0) 0x7fbba4003520 con 0x7fbbc00d2d70 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 msgr2=0x7fbbc00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbbc00d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 >> v1:172.21.15.164:6789/0 conn(0x7fbbc00d3780 legacy=0x7fbbc00d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00b5720 con 0x7fbbc00d2d70 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fbbac001670 con 0x7fbbc00d2d70 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 >> v1:172.21.15.43:6789/0 conn(0x7fbbc00d2d70 legacy=0x7fbbc00cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbba40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004060 0x7fbba4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba4004d40 0x7fbba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00b5d70 con 0x7fbba4004d40 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbba400bc00 con 0x7fbba4004060 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbba400be00 con 0x7fbbc00d31d0 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.298+0000 7fbbbf400640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba4004d40 0x7fbba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbea00640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbba40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbe000640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004060 0x7fbba4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbf400640 1 -- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 msgr2=0x7fbba40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbf400640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d31d0 0x7fbba40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbf400640 1 -- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004060 msgr2=0x7fbba4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbf400640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004060 0x7fbba4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbf400640 1 -- 172.21.15.43:0/2518463504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba4003520 con 0x7fbba4004d40 2024-04-26T20:27:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.299+0000 7fbbbe000640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004060 0x7fbba4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.300+0000 7fbbbf400640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba4004d40 0x7fbba40051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8007330 tx=0x7fbba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662fba5bf62aa3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.300+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8033070 con 0x7fbba4004d40 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.300+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb802f650 con 0x7fbba4004d40 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.300+0000 7fbbbd600640 1 -- 172.21.15.43:0/2518463504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8009be0 con 0x7fbba4004d40 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 -- 172.21.15.43:0/2518463504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba4004d40 msgr2=0x7fbba40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 --2- 172.21.15.43:0/2518463504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba4004d40 0x7fbba40051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8007330 tx=0x7fbba4005830 comp rx=0 tx=0).stop 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbbf400640 1 -- 172.21.15.43:0/2518463504 reap_dead start 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 -- 172.21.15.43:0/2518463504 shutdown_connections 2024-04-26T20:27:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 -- 172.21.15.43:0/2518463504 >> 172.21.15.43:0/2518463504 conn(0x7fbbc008ec50 msgr2=0x7fbbc009e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 -- 172.21.15.43:0/2518463504 shutdown_connections 2024-04-26T20:27:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.301+0000 7fbbc5600640 1 -- 172.21.15.43:0/2518463504 wait complete. 2024-04-26T20:27:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.303+0000 7fbbc5600640 1 Processor -- start 2024-04-26T20:27:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.303+0000 7fbbc5600640 1 -- start start 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004d40 0x7fbbc0161920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d0be0 0x7fbbc0159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00b4920 con 0x7fbbc00d1060 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc015f5e0 con 0x7fbba4004d40 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbc5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc00b4aa0 con 0x7fbbc00d0be0 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbbf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57412/0 (socket says 172.21.15.43:57412) 2024-04-26T20:27:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.304+0000 7fbbbf400640 1 -- 172.21.15.43:0/1713429628 learned_addr learned my addr 172.21.15.43:0/1713429628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:04.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbe000640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d0be0 0x7fbbc0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbea00640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004d40 0x7fbbc0161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 -- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d0be0 msgr2=0x7fbbc0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d0be0 0x7fbbc0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 -- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004d40 msgr2=0x7fbbc0161920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004d40 0x7fbbc0161920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 -- 172.21.15.43:0/1713429628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00cc310 con 0x7fbbc00d1060 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbbf400640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbbb802b030 tx=0x7fbbb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.305+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8033040 con 0x7fbbc00d1060 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.306+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb802f450 con 0x7fbbc00d1060 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.306+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8009be0 con 0x7fbbc00d1060 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.306+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc00cc580 con 0x7fbbc00d1060 2024-04-26T20:27:04.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.306+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc00ccb40 con 0x7fbbc00d1060 2024-04-26T20:27:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.307+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc0069350 con 0x7fbbc00d1060 2024-04-26T20:27:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.308+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbb8040030 con 0x7fbbc00d1060 2024-04-26T20:27:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.309+0000 7fbbb6a00640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 0x7fbba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.309+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fbbb80ac590 con 0x7fbbc00d1060 2024-04-26T20:27:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.309+0000 7fbbbea00640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 0x7fbba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.310+0000 7fbbbea00640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 0x7fbba006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0007fd0 tx=0x7fbbb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.313+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbbb8047030 con 0x7fbbc00d1060 2024-04-26T20:27:04.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.522+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbc00b2300 con 0x7fbbc00d1060 2024-04-26T20:27:04.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.523+0000 7fbbb6a00640 1 -- 172.21.15.43:0/1713429628 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fbbb807a4f0 con 0x7fbbc00d1060 2024-04-26T20:27:04.523 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:04.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.525+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 msgr2=0x7fbba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.525+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 0x7fbba006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0007fd0 tx=0x7fbbb0009040 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.525+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 msgr2=0x7fbbc0159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.525+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbbb802b030 tx=0x7fbbb8004ab0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 shutdown_connections 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbba006c8f0 0x7fbba006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc00d0be0 0x7fbbc0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbc00d1060 0x7fbbc0159b60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 --2- 172.21.15.43:0/1713429628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbba4004d40 0x7fbbc0161920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 >> 172.21.15.43:0/1713429628 conn(0x7fbbc008ec50 msgr2=0x7fbbc00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 shutdown_connections 2024-04-26T20:27:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.526+0000 7fbbc5600640 1 -- 172.21.15.43:0/1713429628 wait complete. 2024-04-26T20:27:04.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.766+0000 7f437b200640 1 Processor -- start 2024-04-26T20:27:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.767+0000 7f437b200640 1 -- start start 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f437b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43740d12e0 0x7f43740b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f437b200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437415f480 con 0x7f43740b5be0 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f437b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437415f760 con 0x7f43740b5fc0 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f437b200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4374160090 con 0x7f43740d12e0 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f4379400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43740d12e0 0x7f43740b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f4379e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f43740b5be0 0x7f43740d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41212/0 (socket says 172.21.15.43:41212) 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.768+0000 7f4379e00640 1 -- 172.21.15.43:0/743553442 learned_addr learned my addr 172.21.15.43:0/743553442 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:04.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.769+0000 7f4379400640 1 -- 172.21.15.43:0/743553442 >> v1:172.21.15.164:6789/0 conn(0x7f43740b5fc0 legacy=0x7f437415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:04.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.769+0000 7f4379400640 1 -- 172.21.15.43:0/743553442 >> v1:172.21.15.43:6789/0 conn(0x7f43740b5be0 legacy=0x7f43740d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.769+0000 7f4379400640 1 -- 172.21.15.43:0/743553442 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437415fa40 con 0x7f43740d12e0 2024-04-26T20:27:04.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.770+0000 7f4379400640 1 --2- 172.21.15.43:0/743553442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f43740b4100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43640097a0 tx=0x7f436402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ef67116316a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f4378a00640 1 -- 172.21.15.43:0/743553442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364037070 con 0x7f43740d12e0 2024-04-26T20:27:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f4378a00640 1 -- 172.21.15.43:0/743553442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364032440 con 0x7f43740d12e0 2024-04-26T20:27:04.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f4378a00640 1 -- 172.21.15.43:0/743553442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364032810 con 0x7f43740d12e0 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 -- 172.21.15.43:0/743553442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 msgr2=0x7f43740b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 --2- 172.21.15.43:0/743553442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f43740b4100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43640097a0 tx=0x7f436402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 -- 172.21.15.43:0/743553442 shutdown_connections 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 --2- 172.21.15.43:0/743553442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f43740b4100 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 -- 172.21.15.43:0/743553442 >> 172.21.15.43:0/743553442 conn(0x7f43740684e0 msgr2=0x7f43740688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 -- 172.21.15.43:0/743553442 shutdown_connections 2024-04-26T20:27:04.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.771+0000 7f437b200640 1 -- 172.21.15.43:0/743553442 wait complete. 2024-04-26T20:27:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.773+0000 7f437b200640 1 Processor -- start 2024-04-26T20:27:04.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.773+0000 7f437b200640 1 -- start start 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43740b5be0 0x7f437414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43740b5fc0 0x7f437414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f437415cc60 con 0x7f43740d12e0 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f437415cae0 con 0x7f43740b5fc0 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.774+0000 7f437b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4374169c60 con 0x7f43740b5be0 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57426/0 (socket says 172.21.15.43:57426) 2024-04-26T20:27:04.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 -- 172.21.15.43:0/3340588475 learned_addr learned my addr 172.21.15.43:0/3340588475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f4379400640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43740b5fc0 0x7f437414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f4379e00640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43740b5be0 0x7f437414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 -- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43740b5be0 msgr2=0x7f437414f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43740b5be0 0x7f437414f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 -- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43740b5fc0 msgr2=0x7f437414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43740b5fc0 0x7f437414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.775+0000 7f437a800640 1 -- 172.21.15.43:0/3340588475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437415b7d0 con 0x7f43740d12e0 2024-04-26T20:27:04.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f437a800640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f437000b970 tx=0x7f437000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370019070 con 0x7f43740d12e0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437000cb40 con 0x7f43740d12e0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370014440 con 0x7f43740d12e0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4374151f50 con 0x7f43740d12e0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.776+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4374152510 con 0x7f43740d12e0 2024-04-26T20:27:04.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.778+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43740506c0 con 0x7f43740d12e0 2024-04-26T20:27:04.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.779+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f437001c070 con 0x7f43740d12e0 2024-04-26T20:27:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.780+0000 7f436e000640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 0x7f435806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:04.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.780+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f437008bb70 con 0x7f43740d12e0 2024-04-26T20:27:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.780+0000 7f4379e00640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 0x7f435806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:04.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.781+0000 7f4379e00640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 0x7f435806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4368000f60 tx=0x7f4368009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:04.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.785+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4370059b60 con 0x7f43740d12e0 2024-04-26T20:27:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.990+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f437406cb40 con 0x7f43740d12e0 2024-04-26T20:27:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.991+0000 7f436e000640 1 -- 172.21.15.43:0/3340588475 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f4370059460 con 0x7f43740d12e0 2024-04-26T20:27:04.992 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 msgr2=0x7f435806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 0x7f435806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4368000f60 tx=0x7f4368009040 comp rx=0 tx=0).stop 2024-04-26T20:27:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 msgr2=0x7f4374152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f437000b970 tx=0x7f437000be30 comp rx=0 tx=0).stop 2024-04-26T20:27:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 shutdown_connections 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f435806c8f0 0x7f435806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43740d12e0 0x7f4374152ff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43740b5fc0 0x7f437414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 --2- 172.21.15.43:0/3340588475 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f43740b5be0 0x7f437414f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.994+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 >> 172.21.15.43:0/3340588475 conn(0x7f43740684e0 msgr2=0x7f43740b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.995+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 shutdown_connections 2024-04-26T20:27:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:04.995+0000 7f437b200640 1 -- 172.21.15.43:0/3340588475 wait complete. 2024-04-26T20:27:05.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:05.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.221+0000 7fe5c7e00640 1 Processor -- start 2024-04-26T20:27:05.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.222+0000 7fe5c7e00640 1 -- start start 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.223+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.223+0000 7fe5c7e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5c80b4d20 0x7fe5c806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.223+0000 7fe5c7e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c815c2a0 con 0x7fe5c815bd60 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.223+0000 7fe5c7e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c815d060 con 0x7fe5c80b4d20 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.223+0000 7fe5c7e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c815c580 con 0x7fe5c815b980 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5c80b4d20 0x7fe5c806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5c80b4d20 0x7fe5c806f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57438/0 (socket says 172.21.15.43:57438) 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 -- 172.21.15.43:0/261930381 learned_addr learned my addr 172.21.15.43:0/261930381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c6a00640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 -- 172.21.15.43:0/261930381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 msgr2=0x7fe5c806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 -- 172.21.15.43:0/261930381 >> v1:172.21.15.164:6789/0 conn(0x7fe5c815bd60 legacy=0x7fe5c80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.224+0000 7fe5c7400640 1 -- 172.21.15.43:0/261930381 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c815c860 con 0x7fe5c80b4d20 2024-04-26T20:27:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.225+0000 7fe5c7400640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c806f1d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009e50 tx=0x7fe5c002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beba3d05ef128690 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.226+0000 7fe5c5600640 1 -- 172.21.15.43:0/261930381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0039070 con 0x7fe5c80b4d20 2024-04-26T20:27:05.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.226+0000 7fe5c5600640 1 -- 172.21.15.43:0/261930381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c0034680 con 0x7fe5c80b4d20 2024-04-26T20:27:05.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.226+0000 7fe5c5600640 1 -- 172.21.15.43:0/261930381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0034a50 con 0x7fe5c80b4d20 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.226+0000 7fe5c7e00640 1 -- 172.21.15.43:0/261930381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 msgr2=0x7fe5c806f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.226+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c806f1d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009e50 tx=0x7fe5c002c8d0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 -- 172.21.15.43:0/261930381 shutdown_connections 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c806f1d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/261930381 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 -- 172.21.15.43:0/261930381 >> 172.21.15.43:0/261930381 conn(0x7fe5c8068090 msgr2=0x7fe5c80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 -- 172.21.15.43:0/261930381 shutdown_connections 2024-04-26T20:27:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.227+0000 7fe5c7e00640 1 -- 172.21.15.43:0/261930381 wait complete. 2024-04-26T20:27:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.229+0000 7fe5c7e00640 1 Processor -- start 2024-04-26T20:27:05.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.229+0000 7fe5c7e00640 1 -- start start 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c81f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 0x7fe5c8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c806db80 con 0x7fe5c80b4d20 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80b52b0 con 0x7fe5c815bd60 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c806dd00 con 0x7fe5c815b980 2024-04-26T20:27:05.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57450/0 (socket says 172.21.15.43:57450) 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c6a00640 1 -- 172.21.15.43:0/2994753737 learned_addr learned my addr 172.21.15.43:0/2994753737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c6000640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c81f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.230+0000 7fe5c7400640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 0x7fe5c8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 -- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 msgr2=0x7fe5c81f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c81f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 -- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 msgr2=0x7fe5c8158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 0x7fe5c8158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 -- 172.21.15.43:0/2994753737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c81f5b30 con 0x7fe5c80b4d20 2024-04-26T20:27:05.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c7400640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 0x7fe5c8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:05.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c6a00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d830 tx=0x7fe5b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800b6c0 con 0x7fe5c80b4d20 2024-04-26T20:27:05.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b8019070 con 0x7fe5c80b4d20 2024-04-26T20:27:05.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800ba80 con 0x7fe5c80b4d20 2024-04-26T20:27:05.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c80c6610 con 0x7fe5c80b4d20 2024-04-26T20:27:05.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.231+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c80cf9b0 con 0x7fe5c80b4d20 2024-04-26T20:27:05.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.233+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c80506c0 con 0x7fe5c80b4d20 2024-04-26T20:27:05.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.234+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5b801e460 con 0x7fe5c80b4d20 2024-04-26T20:27:05.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.235+0000 7fe5bea00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.235+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fe5b808c670 con 0x7fe5c80b4d20 2024-04-26T20:27:05.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.235+0000 7fe5c6000640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.236+0000 7fe5c6000640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 0x7fe5a406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006fd0 tx=0x7fe5b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.240+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5c81f5b30 con 0x7fe5c80b4d20 2024-04-26T20:27:05.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.449+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe5c80af9c0 con 0x7fe5c80b4d20 2024-04-26T20:27:05.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.450+0000 7fe5bea00640 1 -- 172.21.15.43:0/2994753737 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe5b805a660 con 0x7fe5c80b4d20 2024-04-26T20:27:05.451 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:05.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 msgr2=0x7fe5a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 0x7fe5a406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006fd0 tx=0x7fe5b0008040 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 msgr2=0x7fe5c81f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d830 tx=0x7fe5b800dcf0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 shutdown_connections 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe5a406c8f0 0x7fe5a406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe5c815b980 0x7fe5c81f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe5c815bd60 0x7fe5c8158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 --2- 172.21.15.43:0/2994753737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5c80b4d20 0x7fe5c81f5d30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 >> 172.21.15.43:0/2994753737 conn(0x7fe5c8068090 msgr2=0x7fe5c815aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 shutdown_connections 2024-04-26T20:27:05.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.453+0000 7fe5c7e00640 1 -- 172.21.15.43:0/2994753737 wait complete. 2024-04-26T20:27:05.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-26T20:27:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.680+0000 7fe1b7200640 1 Processor -- start 2024-04-26T20:27:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.680+0000 7fe1b7200640 1 -- start start 2024-04-26T20:27:05.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1b00d3480 0x7fe1b00c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b0162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b7200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b00cf860 con 0x7fe1b00b5ca0 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b00d3090 con 0x7fe1b00d3480 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b00d5b50 con 0x7fe1b00d1d90 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b6800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe1b00b5ca0 0x7fe1b00cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41222/0 (socket says 172.21.15.43:41222) 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b6800640 1 -- 172.21.15.43:0/357789527 learned_addr learned my addr 172.21.15.43:0/357789527 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.682+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/357789527 >> v2:172.21.15.43:3300/0 conn(0x7fe1b00d3480 0x7fe1b00c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b5400640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b0162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b4a00640 1 -- 172.21.15.43:0/357789527 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3226495598 0 0) 0x7fe1b00cf860 con 0x7fe1b00b5ca0 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b4a00640 1 -- 172.21.15.43:0/357789527 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe198003520 con 0x7fe1b00b5ca0 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b5e00640 1 -- 172.21.15.43:0/357789527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 msgr2=0x7fe1b0162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:05.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b0162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b5e00640 1 -- 172.21.15.43:0/357789527 >> v1:172.21.15.43:6789/0 conn(0x7fe1b00b5ca0 legacy=0x7fe1b00cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.683+0000 7fe1b5e00640 1 -- 172.21.15.43:0/357789527 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b00cfb00 con 0x7fe1b00d3480 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.684+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00c87a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40097a0 tx=0x7fe1a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeadc6380ad6d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b4a00640 1 -- 172.21.15.43:0/357789527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4037070 con 0x7fe1b00d3480 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b4a00640 1 -- 172.21.15.43:0/357789527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a4032440 con 0x7fe1b00d3480 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b4a00640 1 -- 172.21.15.43:0/357789527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4032810 con 0x7fe1b00d3480 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 -- 172.21.15.43:0/357789527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 msgr2=0x7fe1b00c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00c87a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40097a0 tx=0x7fe1a402a180 comp rx=0 tx=0).stop 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 -- 172.21.15.43:0/357789527 shutdown_connections 2024-04-26T20:27:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b0162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 --2- 172.21.15.43:0/357789527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00c87a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 -- 172.21.15.43:0/357789527 >> 172.21.15.43:0/357789527 conn(0x7fe1b00d4c50 msgr2=0x7fe1b00d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 -- 172.21.15.43:0/357789527 shutdown_connections 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.685+0000 7fe1b7200640 1 -- 172.21.15.43:0/357789527 wait complete. 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.687+0000 7fe1b7200640 1 Processor -- start 2024-04-26T20:27:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.687+0000 7fe1b7200640 1 -- start start 2024-04-26T20:27:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b00b5ca0 0x7fe1b01f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b01fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b00d5770 con 0x7fe1b00d3480 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b00d2db0 con 0x7fe1b00b5ca0 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.688+0000 7fe1b7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b00d58f0 con 0x7fe1b00d1d90 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57472/0 (socket says 172.21.15.43:57472) 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 -- 172.21.15.43:0/2300466566 learned_addr learned my addr 172.21.15.43:0/2300466566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b5400640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b01fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b00b5ca0 0x7fe1b01f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 -- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 msgr2=0x7fe1b01fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b01fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 -- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b00b5ca0 msgr2=0x7fe1b01f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b00b5ca0 0x7fe1b01f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 -- 172.21.15.43:0/2300466566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b01f9be0 con 0x7fe1b00d3480 2024-04-26T20:27:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b5400640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b01fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.689+0000 7fe1b6800640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00cb70 tx=0x7fe1ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.690+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac017070 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.690+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac00ad40 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.690+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac012620 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.690+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b00cbf30 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.690+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b00ceba0 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.691+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b00cbd20 con 0x7fe1b00d3480 2024-04-26T20:27:05.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.692+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1ac01a070 con 0x7fe1b00d3480 2024-04-26T20:27:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.693+0000 7fe1aa000640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.693+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fe1ac08bdf0 con 0x7fe1b00d3480 2024-04-26T20:27:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.694+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.694+0000 7fe1b5e00640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4034000 tx=0x7fe1a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:05.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.698+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1ac059df0 con 0x7fe1b00d3480 2024-04-26T20:27:05.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:05.978+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fe1b0069350 con 0x7fe1b00d3480 2024-04-26T20:27:06.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.445+0000 7fe1aa000640 1 -- 172.21.15.43:0/2300466566 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v81) ==== 123+0+0 (secure 0 0 0) 0x7fe1ac0596f0 con 0x7fe1b00d3480 2024-04-26T20:27:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.448+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 msgr2=0x7fe19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.448+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4034000 tx=0x7fe1a402e000 comp rx=0 tx=0).stop 2024-04-26T20:27:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.448+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 msgr2=0x7fe1b00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.448+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00cb70 tx=0x7fe1ac00a430 comp rx=0 tx=0).stop 2024-04-26T20:27:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 shutdown_connections 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1b00d1d90 0x7fe1b01fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b00d3480 0x7fe1b00ce660 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 --2- 172.21.15.43:0/2300466566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1b00b5ca0 0x7fe1b01f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 >> 172.21.15.43:0/2300466566 conn(0x7fe1b00d4c50 msgr2=0x7fe1b01611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 shutdown_connections 2024-04-26T20:27:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.449+0000 7fe1b7200640 1 -- 172.21.15.43:0/2300466566 wait complete. 2024-04-26T20:27:06.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:27:06.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.688+0000 7f0266800640 1 Processor -- start 2024-04-26T20:27:06.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.689+0000 7f0266800640 1 -- start start 2024-04-26T20:27:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.690+0000 7f0266800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.690+0000 7f0266800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 0x7f02600b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.690+0000 7f0266800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600b3900 con 0x7f02600b5410 2024-04-26T20:27:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.690+0000 7f0266800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f026015abc0 con 0x7f026015d9c0 2024-04-26T20:27:06.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.690+0000 7f0266800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f02600b3b70 con 0x7f026015a7e0 2024-04-26T20:27:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.691+0000 7f0265400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.691+0000 7f0265400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:51668/0 (socket says 172.21.15.43:51668) 2024-04-26T20:27:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.691+0000 7f0265400640 1 -- 172.21.15.43:0/3736634393 learned_addr learned my addr 172.21.15.43:0/3736634393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.691+0000 7f0265e00640 1 --2- 172.21.15.43:0/3736634393 >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 0x7f02600b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f025fe00640 1 -- 172.21.15.43:0/3736634393 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008759631 0 0) 0x7f02600b3900 con 0x7f02600b5410 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f025fe00640 1 -- 172.21.15.43:0/3736634393 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0248003520 con 0x7f02600b5410 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f0265400640 1 -- 172.21.15.43:0/3736634393 >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 msgr2=0x7f02600b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f0265400640 1 --2- 172.21.15.43:0/3736634393 >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 0x7f02600b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f0265400640 1 -- 172.21.15.43:0/3736634393 >> v1:172.21.15.164:6789/0 conn(0x7f02600b5410 legacy=0x7f026015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f0265400640 1 -- 172.21.15.43:0/3736634393 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600b3e50 con 0x7f026015a7e0 2024-04-26T20:27:06.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.692+0000 7f0265e00640 1 --2- 172.21.15.43:0/3736634393 >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 0x7f02600b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:06.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.694+0000 7f0265400640 1 --2- 172.21.15.43:0/3736634393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0254009870 tx=0x7f025402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97137f7aa5e13111 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:06.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.694+0000 7f025fe00640 1 -- 172.21.15.43:0/3736634393 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254037070 con 0x7f026015a7e0 2024-04-26T20:27:06.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.694+0000 7f025fe00640 1 -- 172.21.15.43:0/3736634393 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0254032440 con 0x7f026015a7e0 2024-04-26T20:27:06.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.694+0000 7f025fe00640 1 -- 172.21.15.43:0/3736634393 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02540327f0 con 0x7f026015a7e0 2024-04-26T20:27:06.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 -- 172.21.15.43:0/3736634393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 msgr2=0x7f02600b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 --2- 172.21.15.43:0/3736634393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0254009870 tx=0x7f025402a270 comp rx=0 tx=0).stop 2024-04-26T20:27:06.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 -- 172.21.15.43:0/3736634393 shutdown_connections 2024-04-26T20:27:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 --2- 172.21.15.43:0/3736634393 >> v2:172.21.15.164:3300/0 conn(0x7f026015d9c0 0x7f02600b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 --2- 172.21.15.43:0/3736634393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015a7e0 0x7f02600b4650 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 -- 172.21.15.43:0/3736634393 >> 172.21.15.43:0/3736634393 conn(0x7f0260068090 msgr2=0x7f02600684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 -- 172.21.15.43:0/3736634393 shutdown_connections 2024-04-26T20:27:06.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.695+0000 7f0266800640 1 -- 172.21.15.43:0/3736634393 wait complete. 2024-04-26T20:27:06.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.697+0000 7f0266800640 1 Processor -- start 2024-04-26T20:27:06.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.697+0000 7f0266800640 1 -- start start 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026015a7e0 0x7f02600aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015d9c0 0x7f026014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b4b90 con 0x7f02600b5410 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f026015e010 con 0x7f026015a7e0 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0266800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f026015e2e0 con 0x7f026015d9c0 2024-04-26T20:27:06.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0265400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0265400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45156/0 (socket says 172.21.15.43:45156) 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0265400640 1 -- 172.21.15.43:0/1338359089 learned_addr learned my addr 172.21.15.43:0/1338359089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0265e00640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015d9c0 0x7f026014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.698+0000 7f0264a00640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026015a7e0 0x7f02600aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 -- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015d9c0 msgr2=0x7f026014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015d9c0 0x7f026014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 -- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026015a7e0 msgr2=0x7f02600aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026015a7e0 0x7f02600aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 -- 172.21.15.43:0/1338359089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600aba10 con 0x7f02600b5410 2024-04-26T20:27:06.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0265400640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f025402a2b0 tx=0x7f02540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254037040 con 0x7f02600b5410 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0254003bc0 con 0x7f02600b5410 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02600ab170 con 0x7f02600b5410 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.699+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02540325f0 con 0x7f02600b5410 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.700+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02600ab730 con 0x7f02600b5410 2024-04-26T20:27:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.701+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02600aaef0 con 0x7f02600b5410 2024-04-26T20:27:06.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.701+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0254032790 con 0x7f02600b5410 2024-04-26T20:27:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.702+0000 7f025d600640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 0x7f024006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.702+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f02540ac560 con 0x7f02600b5410 2024-04-26T20:27:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.703+0000 7f0264a00640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 0x7f024006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:06.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.703+0000 7f0264a00640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 0x7f024006ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02600b5940 tx=0x7f024c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.707+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02600aaef0 con 0x7f02600b5410 2024-04-26T20:27:06.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.910+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f02600506c0 con 0x7f02600b5410 2024-04-26T20:27:06.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.917+0000 7f025d600640 1 -- 172.21.15.43:0/1338359089 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f025407a540 con 0x7f02600b5410 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 msgr2=0x7f024006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 0x7f024006ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02600b5940 tx=0x7f024c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 msgr2=0x7f02601e8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f025402a2b0 tx=0x7f02540046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 shutdown_connections 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f024006c940 0x7f024006ee00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f026015d9c0 0x7f026014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f026015a7e0 0x7f02600aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 --2- 172.21.15.43:0/1338359089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02600b5410 0x7f02601e8990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:06.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 >> 172.21.15.43:0/1338359089 conn(0x7f0260068090 msgr2=0x7f02600cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:06.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 shutdown_connections 2024-04-26T20:27:06.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:06.920+0000 7f0266800640 1 -- 172.21.15.43:0/1338359089 wait complete. 2024-04-26T20:27:06.941 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:27:06.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:27:07.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:27:07.190 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:27:07.190 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:07.191 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:07.253 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:07.253 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:07.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.512+0000 7fca9f000640 1 Processor -- start 2024-04-26T20:27:07.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.512+0000 7fca9f000640 1 -- start start 2024-04-26T20:27:07.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9f000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca981628e0 0x7fca98162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9f000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca98145710 con 0x7fca98145330 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9f000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca98073dc0 con 0x7fca980af720 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9f000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fca980b0250 con 0x7fca981628e0 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9d200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca981628e0 0x7fca98162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9dc00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca98145330 0x7fca980b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51864/0 (socket says 172.21.15.43:51864) 2024-04-26T20:27:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.514+0000 7fca9dc00640 1 -- 172.21.15.43:0/157955018 learned_addr learned my addr 172.21.15.43:0/157955018 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.515+0000 7fca9d200640 1 -- 172.21.15.43:0/157955018 >> v1:172.21.15.164:6789/0 conn(0x7fca980af720 legacy=0x7fca980afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.515+0000 7fca9d200640 1 -- 172.21.15.43:0/157955018 >> v1:172.21.15.43:6789/0 conn(0x7fca98145330 legacy=0x7fca980b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.515+0000 7fca9d200640 1 -- 172.21.15.43:0/157955018 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980ae960 con 0x7fca981628e0 2024-04-26T20:27:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.515+0000 7fca97e00640 1 -- 172.21.15.43:0/157955018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589066060 0 0) 0x7fca98145710 con 0x7fca98145330 2024-04-26T20:27:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.516+0000 7fca9d200640 1 --2- 172.21.15.43:0/157955018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca981628e0 0x7fca98162d70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca980b03d0 tx=0x7fca8402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1419e8611167254f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.516+0000 7fca97e00640 1 -- 172.21.15.43:0/157955018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84037070 con 0x7fca981628e0 2024-04-26T20:27:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.516+0000 7fca97e00640 1 -- 172.21.15.43:0/157955018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84032440 con 0x7fca981628e0 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.516+0000 7fca97e00640 1 -- 172.21.15.43:0/157955018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84032810 con 0x7fca981628e0 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 -- 172.21.15.43:0/157955018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca981628e0 msgr2=0x7fca98162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 --2- 172.21.15.43:0/157955018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca981628e0 0x7fca98162d70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca980b03d0 tx=0x7fca8402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 -- 172.21.15.43:0/157955018 shutdown_connections 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 --2- 172.21.15.43:0/157955018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca981628e0 0x7fca98162d70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 -- 172.21.15.43:0/157955018 >> 172.21.15.43:0/157955018 conn(0x7fca98068090 msgr2=0x7fca98068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 -- 172.21.15.43:0/157955018 shutdown_connections 2024-04-26T20:27:07.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.517+0000 7fca9f000640 1 -- 172.21.15.43:0/157955018 wait complete. 2024-04-26T20:27:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.519+0000 7fca9f000640 1 Processor -- start 2024-04-26T20:27:07.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.519+0000 7fca9f000640 1 -- start start 2024-04-26T20:27:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 0x7fca980b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca981628e0 0x7fca9813ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca980af200 con 0x7fca98145330 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca980aef20 con 0x7fca980af720 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9f000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fca98073000 con 0x7fca981628e0 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9d200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45176/0 (socket says 172.21.15.43:45176) 2024-04-26T20:27:07.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9d200640 1 -- 172.21.15.43:0/1529349918 learned_addr learned my addr 172.21.15.43:0/1529349918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9dc00640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 0x7fca980b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9e600640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca981628e0 0x7fca9813ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.520+0000 7fca9d200640 1 -- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca981628e0 msgr2=0x7fca9813ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9d200640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca981628e0 0x7fca9813ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9d200640 1 -- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 msgr2=0x7fca980b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9d200640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 0x7fca980b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9d200640 1 -- 172.21.15.43:0/1529349918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980cd410 con 0x7fca98145330 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9dc00640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 0x7fca980b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9d200640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca84009890 tx=0x7fca840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84037040 con 0x7fca98145330 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca84004230 con 0x7fca98145330 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca980cd750 con 0x7fca98145330 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca84032e20 con 0x7fca98145330 2024-04-26T20:27:07.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.521+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca980cdf90 con 0x7fca98145330 2024-04-26T20:27:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.522+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca98069350 con 0x7fca98145330 2024-04-26T20:27:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.524+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca84030770 con 0x7fca98145330 2024-04-26T20:27:07.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.524+0000 7fca95600640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.525+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fca840ad2e0 con 0x7fca98145330 2024-04-26T20:27:07.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.525+0000 7fca9dc00640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.525+0000 7fca9dc00640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 0x7fca7806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0012e0 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.529+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca980cd750 con 0x7fca98145330 2024-04-26T20:27:07.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.737+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca98144900 con 0x7fca98145330 2024-04-26T20:27:07.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.739+0000 7fca95600640 1 -- 172.21.15.43:0/1529349918 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fca8407b2d0 con 0x7fca98145330 2024-04-26T20:27:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:07.740 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 msgr2=0x7fca7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 0x7fca7806edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0012e0 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 msgr2=0x7fca9815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca84009890 tx=0x7fca840046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 shutdown_connections 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fca7806c8f0 0x7fca7806edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fca981628e0 0x7fca9813ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca98145330 0x7fca9815baa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 --2- 172.21.15.43:0/1529349918 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca980af720 0x7fca980b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 >> 172.21.15.43:0/1529349918 conn(0x7fca98068090 msgr2=0x7fca98161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.742+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 shutdown_connections 2024-04-26T20:27:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.743+0000 7fca9f000640 1 -- 172.21.15.43:0/1529349918 wait complete. 2024-04-26T20:27:07.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:07.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:07.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.977+0000 7f19b4a00640 1 Processor -- start 2024-04-26T20:27:07.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.978+0000 7f19b4a00640 1 -- start start 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.979+0000 7f19b4a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f19b00d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.979+0000 7f19b4a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b00d2d70 con 0x7f19b015f180 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.979+0000 7f19b4a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b00d06e0 con 0x7f19b00b3cb0 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.979+0000 7f19b4a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f19b015f560 con 0x7f19b00d3830 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19af400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f19b015f180 0x7f19b00d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51876/0 (socket says 172.21.15.43:51876) 2024-04-26T20:27:07.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19af400640 1 -- 172.21.15.43:0/2705915008 learned_addr learned my addr 172.21.15.43:0/2705915008 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:07.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19a6a00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f19b00d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035826597 0 0) 0x7f19b00d2d70 con 0x7f19b015f180 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1994003520 con 0x7f19b015f180 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2436105069 0 0) 0x7f1994003520 con 0x7f19b015f180 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 msgr2=0x7f19b00d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.980+0000 7f19aea00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f19b00d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 >> v1:172.21.15.164:6789/0 conn(0x7f19b00b3cb0 legacy=0x7f19b00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b00d09c0 con 0x7f19b015f180 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f19a00019f0 con 0x7f19b015f180 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 >> v1:172.21.15.43:6789/0 conn(0x7f19b015f180 legacy=0x7f19b00d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f1994003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f1994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1994004d40 0x7f19940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19b015f560 con 0x7f1994003a50 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f199400bad0 con 0x7f1994004d40 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f199400bcd0 con 0x7f19b00d3830 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19af400640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f1994003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.981+0000 7f19a6a00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f1994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19af400640 1 -- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 msgr2=0x7f1994004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19af400640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00d3830 0x7f1994004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19afe00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1994004d40 0x7f19940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19af400640 1 -- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1994004d40 msgr2=0x7f19940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19af400640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1994004d40 0x7f19940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.982+0000 7f19af400640 1 -- 172.21.15.43:0/2705915008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1994003520 con 0x7f1994003a50 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.983+0000 7f19af400640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f1994003e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19a0002230 tx=0x7f1994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a5ff4575a01536 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.983+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000a4a0 con 0x7f1994003a50 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.983+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a002f070 con 0x7f1994003a50 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.983+0000 7f19aea00640 1 -- 172.21.15.43:0/2705915008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a002bbe0 con 0x7f1994003a50 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 -- 172.21.15.43:0/2705915008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 msgr2=0x7f1994003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2705915008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f1994003e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f19a0002230 tx=0x7f1994005830 comp rx=0 tx=0).stop 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19afe00640 1 -- 172.21.15.43:0/2705915008 reap_dead start 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 -- 172.21.15.43:0/2705915008 shutdown_connections 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 -- 172.21.15.43:0/2705915008 >> 172.21.15.43:0/2705915008 conn(0x7f19b00d4bd0 msgr2=0x7f19b00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 -- 172.21.15.43:0/2705915008 shutdown_connections 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.984+0000 7f19b4a00640 1 -- 172.21.15.43:0/2705915008 wait complete. 2024-04-26T20:27:07.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.986+0000 7f19b4a00640 1 Processor -- start 2024-04-26T20:27:07.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.986+0000 7f19b4a00640 1 -- start start 2024-04-26T20:27:07.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19b00b9fc0 0x7f19b0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00ba440 0x7f19b0157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19b00d11c0 con 0x7f1994003a50 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f19b00d2a40 con 0x7f19b00b9fc0 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19b4a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f19b00b8fb0 con 0x7f19b00ba440 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19af400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45208/0 (socket says 172.21.15.43:45208) 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.987+0000 7f19af400640 1 -- 172.21.15.43:0/2497477662 learned_addr learned my addr 172.21.15.43:0/2497477662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19a6a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19b00b9fc0 0x7f19b0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 -- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00ba440 msgr2=0x7f19b0157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00ba440 0x7f19b0157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 -- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19b00b9fc0 msgr2=0x7f19b0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19b00b9fc0 0x7f19b0157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 -- 172.21.15.43:0/2497477662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b00b72b0 con 0x7f1994003a50 2024-04-26T20:27:07.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19af400640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a880 tx=0x7f19a0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0044070 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.988+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a0001250 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.989+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0004dc0 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.989+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b00b83a0 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.989+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b00b8960 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.990+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b00506c0 con 0x7f1994003a50 2024-04-26T20:27:07.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.991+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19a0034470 con 0x7f1994003a50 2024-04-26T20:27:07.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.992+0000 7f19a7e00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:07.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.992+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f19a00abdc0 con 0x7f1994003a50 2024-04-26T20:27:07.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.992+0000 7f19a6a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:07.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.993+0000 7f19a6a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 0x7f198c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19b00b98a0 tx=0x7f1998008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:07.996+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19a0001ab0 con 0x7f1994003a50 2024-04-26T20:27:08.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.200+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19b00b2990 con 0x7f1994003a50 2024-04-26T20:27:08.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.201+0000 7f19a7e00640 1 -- 172.21.15.43:0/2497477662 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f19a0079db0 con 0x7f1994003a50 2024-04-26T20:27:08.201 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 msgr2=0x7f198c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 0x7f198c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f19b00b98a0 tx=0x7f1998008040 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 msgr2=0x7f19b00b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a880 tx=0x7f19a0001dc0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 shutdown_connections 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f19b00ba440 0x7f19b0157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f19b00b9fc0 0x7f19b0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 --2- 172.21.15.43:0/2497477662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1994003a50 0x7f19b00b6d70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 >> 172.21.15.43:0/2497477662 conn(0x7f19b00d4bd0 msgr2=0x7f19b00b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:08.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 shutdown_connections 2024-04-26T20:27:08.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.203+0000 7f19b4a00640 1 -- 172.21.15.43:0/2497477662 wait complete. 2024-04-26T20:27:08.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.441+0000 7fe0c7200640 1 Processor -- start 2024-04-26T20:27:08.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.442+0000 7fe0c7200640 1 -- start start 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.443+0000 7fe0c7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0c80b5410 0x7fe0c80b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.443+0000 7fe0c7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe0c815d2b0 0x7fe0c815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.443+0000 7fe0c7200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c80b3900 con 0x7fe0c815a7e0 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.443+0000 7fe0c7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c815abc0 con 0x7fe0c80b5410 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.443+0000 7fe0c7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c80b3b70 con 0x7fe0c815d2b0 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0c815a7e0 0x7fe0c80b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51892/0 (socket says 172.21.15.43:51892) 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5e00640 1 -- 172.21.15.43:0/2356112643 learned_addr learned my addr 172.21.15.43:0/2356112643 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5400640 1 --2- 172.21.15.43:0/2356112643 >> v2:172.21.15.43:3300/0 conn(0x7fe0c80b5410 0x7fe0c80b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c6800640 1 --2- 172.21.15.43:0/2356112643 >> v2:172.21.15.164:3300/0 conn(0x7fe0c815d2b0 0x7fe0c815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5400640 1 -- 172.21.15.43:0/2356112643 >> v2:172.21.15.164:3300/0 conn(0x7fe0c815d2b0 msgr2=0x7fe0c815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5400640 1 --2- 172.21.15.43:0/2356112643 >> v2:172.21.15.164:3300/0 conn(0x7fe0c815d2b0 0x7fe0c815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5400640 1 -- 172.21.15.43:0/2356112643 >> v1:172.21.15.43:6789/0 conn(0x7fe0c815a7e0 legacy=0x7fe0c80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c5400640 1 -- 172.21.15.43:0/2356112643 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c80b3e50 con 0x7fe0c80b5410 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.444+0000 7fe0c4a00640 1 -- 172.21.15.43:0/2356112643 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477916127 0 0) 0x7fe0c80b3900 con 0x7fe0c815a7e0 2024-04-26T20:27:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.445+0000 7fe0c5400640 1 --2- 172.21.15.43:0/2356112643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c80b5410 0x7fe0c80b5890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80cd1c0 tx=0x7fe0b002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357556877f3313cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.446+0000 7fe0c4a00640 1 -- 172.21.15.43:0/2356112643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0037070 con 0x7fe0c80b5410 2024-04-26T20:27:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.446+0000 7fe0c4a00640 1 -- 172.21.15.43:0/2356112643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0032440 con 0x7fe0c80b5410 2024-04-26T20:27:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.446+0000 7fe0c4a00640 1 -- 172.21.15.43:0/2356112643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0032810 con 0x7fe0c80b5410 2024-04-26T20:27:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.446+0000 7fe0c7200640 1 -- 172.21.15.43:0/2356112643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c80b5410 msgr2=0x7fe0c80b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.446+0000 7fe0c7200640 1 --2- 172.21.15.43:0/2356112643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c80b5410 0x7fe0c80b5890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80cd1c0 tx=0x7fe0b002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 -- 172.21.15.43:0/2356112643 shutdown_connections 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 --2- 172.21.15.43:0/2356112643 >> v2:172.21.15.164:3300/0 conn(0x7fe0c815d2b0 0x7fe0c815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 --2- 172.21.15.43:0/2356112643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c80b5410 0x7fe0c80b5890 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 -- 172.21.15.43:0/2356112643 >> 172.21.15.43:0/2356112643 conn(0x7fe0c8068090 msgr2=0x7fe0c80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 -- 172.21.15.43:0/2356112643 shutdown_connections 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.447+0000 7fe0c7200640 1 -- 172.21.15.43:0/2356112643 wait complete. 2024-04-26T20:27:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.448+0000 7fe0c7200640 1 Processor -- start 2024-04-26T20:27:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.449+0000 7fe0c7200640 1 -- start start 2024-04-26T20:27:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 0x7fe0c8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 0x7fe0c80ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815dc80 con 0x7fe0c815d2b0 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815e010 con 0x7fe0c815a7e0 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c815e2e0 con 0x7fe0c80b5410 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45230/0 (socket says 172.21.15.43:45230) 2024-04-26T20:27:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c6800640 1 -- 172.21.15.43:0/398901695 learned_addr learned my addr 172.21.15.43:0/398901695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:08.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c5400640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 0x7fe0c80ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c6800640 1 -- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 msgr2=0x7fe0c8150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.450+0000 7fe0c5e00640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 0x7fe0c8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c6800640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 0x7fe0c8150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c6800640 1 -- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 msgr2=0x7fe0c80ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c6800640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 0x7fe0c80ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c6800640 1 -- 172.21.15.43:0/398901695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8073a40 con 0x7fe0c815d2b0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c5400640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 0x7fe0c80ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c5e00640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 0x7fe0c8150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c6800640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007e60 tx=0x7fe0c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0014420 con 0x7fe0c815d2b0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c0019070 con 0x7fe0c815d2b0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c80c9be0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.451+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0014800 con 0x7fe0c815d2b0 2024-04-26T20:27:08.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.452+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c80ca1a0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.453+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c80506c0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.454+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0c00149a0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.455+0000 7fe0be000640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.455+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fe0c008c7e0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.455+0000 7fe0c5e00640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.456+0000 7fe0c5e00640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 0x7fe0a406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000f60 tx=0x7fe0b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.460+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0c005a7e0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.664+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0c8159fe0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.665+0000 7fe0be000640 1 -- 172.21.15.43:0/398901695 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe0c005a0e0 con 0x7fe0c815d2b0 2024-04-26T20:27:08.666 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:08.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 msgr2=0x7fe0a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 0x7fe0a406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8000f60 tx=0x7fe0b8009040 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 msgr2=0x7fe0c8072ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007e60 tx=0x7fe0c000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 shutdown_connections 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c815d2b0 0x7fe0c8072ed0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c815a7e0 0x7fe0c80ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 --2- 172.21.15.43:0/398901695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0c80b5410 0x7fe0c8150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 >> 172.21.15.43:0/398901695 conn(0x7fe0c8068090 msgr2=0x7fe0c815c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 shutdown_connections 2024-04-26T20:27:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.668+0000 7fe0c7200640 1 -- 172.21.15.43:0/398901695 wait complete. 2024-04-26T20:27:08.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:08.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.904+0000 7f7e1be00640 1 Processor -- start 2024-04-26T20:27:08.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.905+0000 7f7e1be00640 1 -- start start 2024-04-26T20:27:08.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.906+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 0x7f7e1c0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.906+0000 7f7e1be00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.906+0000 7f7e1be00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e1c0d0490 con 0x7f7e1c15f180 2024-04-26T20:27:08.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.906+0000 7f7e1be00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1c0d1340 con 0x7f7e1c0cd000 2024-04-26T20:27:08.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.906+0000 7f7e1be00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1c0d0730 con 0x7f7e1c0d1cd0 2024-04-26T20:27:08.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45238/0 (socket says 172.21.15.43:45238) 2024-04-26T20:27:08.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 -- 172.21.15.43:0/2881576382 learned_addr learned my addr 172.21.15.43:0/2881576382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 0x7f7e1c0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 -- 172.21.15.43:0/2881576382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 msgr2=0x7f7e1c0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 0x7f7e1c0d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 -- 172.21.15.43:0/2881576382 >> v1:172.21.15.164:6789/0 conn(0x7f7e1c15f180 legacy=0x7f7e1c0d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e1b400640 1 -- 172.21.15.43:0/2881576382 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c0d0a10 con 0x7f7e1c0cd000 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.907+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 0x7f7e1c0d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.909+0000 7f7e1b400640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009e50 tx=0x7f7e1402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304764a87d260707 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.909+0000 7f7e1a000640 1 -- 172.21.15.43:0/2881576382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14036070 con 0x7f7e1c0cd000 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.909+0000 7f7e1a000640 1 -- 172.21.15.43:0/2881576382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e14031600 con 0x7f7e1c0cd000 2024-04-26T20:27:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.909+0000 7f7e1a000640 1 -- 172.21.15.43:0/2881576382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140319d0 con 0x7f7e1c0cd000 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 -- 172.21.15.43:0/2881576382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 msgr2=0x7f7e1c0d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009e50 tx=0x7f7e1402a640 comp rx=0 tx=0).stop 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 -- 172.21.15.43:0/2881576382 shutdown_connections 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0d2ed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2881576382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c0d1cd0 0x7f7e1c0d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 -- 172.21.15.43:0/2881576382 >> 172.21.15.43:0/2881576382 conn(0x7f7e1c0d4bb0 msgr2=0x7f7e1c0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:08.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 -- 172.21.15.43:0/2881576382 shutdown_connections 2024-04-26T20:27:08.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.910+0000 7f7e1be00640 1 -- 172.21.15.43:0/2881576382 wait complete. 2024-04-26T20:27:08.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.912+0000 7f7e1be00640 1 Processor -- start 2024-04-26T20:27:08.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.912+0000 7f7e1be00640 1 -- start start 2024-04-26T20:27:08.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 0x7f7e1c1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c15f180 0x7f7e1c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0d57d0 con 0x7f7e1c0cd000 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c0d0fd0 con 0x7f7e1c0d1cd0 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1be00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1c0d5950 con 0x7f7e1c15f180 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45246/0 (socket says 172.21.15.43:45246) 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1aa00640 1 -- 172.21.15.43:0/2087855834 learned_addr learned my addr 172.21.15.43:0/2087855834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.913+0000 7f7e1b400640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c15f180 0x7f7e1c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 0x7f7e1c1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 -- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c15f180 msgr2=0x7f7e1c0b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c15f180 0x7f7e1c0b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 -- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 msgr2=0x7f7e1c1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 0x7f7e1c1f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 -- 172.21.15.43:0/2087855834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c0cb5b0 con 0x7f7e1c0cd000 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 0x7f7e1c1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1aa00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00b420 tx=0x7f7e0c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c011830 con 0x7f7e1c0cd000 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0c016070 con 0x7f7e1c0cd000 2024-04-26T20:27:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c011c10 con 0x7f7e1c0cd000 2024-04-26T20:27:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.914+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c0cb890 con 0x7f7e1c0cd000 2024-04-26T20:27:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.915+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c0b9030 con 0x7f7e1c0cd000 2024-04-26T20:27:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.916+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c0506c0 con 0x7f7e1c0cd000 2024-04-26T20:27:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.917+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e0c01b460 con 0x7f7e1c0cd000 2024-04-26T20:27:08.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.918+0000 7f7e12a00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 0x7f7df806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:08.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.918+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f7e0c08bac0 con 0x7f7e1c0cd000 2024-04-26T20:27:08.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.918+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 0x7f7df806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:08.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.919+0000 7f7e13e00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 0x7f7df806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006fd0 tx=0x7f7e04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:08.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:08.923+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e0c020020 con 0x7f7e1c0cd000 2024-04-26T20:27:09.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.133+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e1c06d700 con 0x7f7e1c0cd000 2024-04-26T20:27:09.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.134+0000 7f7e12a00640 1 -- 172.21.15.43:0/2087855834 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f7e0c059ab0 con 0x7f7e1c0cd000 2024-04-26T20:27:09.134 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.136+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 msgr2=0x7f7df806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.136+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 0x7f7df806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006fd0 tx=0x7f7e04008040 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.136+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 msgr2=0x7f7e1c0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.136+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00b420 tx=0x7f7e0c00b8e0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 shutdown_connections 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7df806c8f0 0x7f7df806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7e1c15f180 0x7f7e1c0b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7e1c0d1cd0 0x7f7e1c1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 --2- 172.21.15.43:0/2087855834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e1c0cd000 0x7f7e1c0caf40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 >> 172.21.15.43:0/2087855834 conn(0x7f7e1c0d4bb0 msgr2=0x7f7e1c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 shutdown_connections 2024-04-26T20:27:09.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.137+0000 7f7e1be00640 1 -- 172.21.15.43:0/2087855834 wait complete. 2024-04-26T20:27:09.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:09.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.368+0000 7f6e70a00640 1 Processor -- start 2024-04-26T20:27:09.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.368+0000 7f6e70a00640 1 -- start start 2024-04-26T20:27:09.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e70a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e70a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e6c0b3900 con 0x7f6e6c0b5410 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e70a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6e6c15abc0 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e70a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c0b3b70 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e6b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e6b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45252/0 (socket says 172.21.15.43:45252) 2024-04-26T20:27:09.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.370+0000 7f6e6b200640 1 -- 172.21.15.43:0/1729803338 learned_addr learned my addr 172.21.15.43:0/1729803338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.371+0000 7f6e6a800640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.371+0000 7f6e6b200640 1 -- 172.21.15.43:0/1729803338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15a7e0 msgr2=0x7f6e6c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.371+0000 7f6e6b200640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.371+0000 7f6e6b200640 1 -- 172.21.15.43:0/1729803338 >> v1:172.21.15.164:6789/0 conn(0x7f6e6c0b5410 legacy=0x7f6e6c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:09.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.371+0000 7f6e6b200640 1 -- 172.21.15.43:0/1729803338 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c0b3e50 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.372+0000 7f6e6b200640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47856bee72cd8e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.372+0000 7f6e69400640 1 -- 172.21.15.43:0/1729803338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64036070 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.372+0000 7f6e69400640 1 -- 172.21.15.43:0/1729803338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e64031600 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.372+0000 7f6e69400640 1 -- 172.21.15.43:0/1729803338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e640319d0 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 -- 172.21.15.43:0/1729803338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15d9c0 msgr2=0x7f6e6c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 -- 172.21.15.43:0/1729803338 shutdown_connections 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15d9c0 0x7f6e6c0b5890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 --2- 172.21.15.43:0/1729803338 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 -- 172.21.15.43:0/1729803338 >> 172.21.15.43:0/1729803338 conn(0x7f6e6c068090 msgr2=0x7f6e6c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 -- 172.21.15.43:0/1729803338 shutdown_connections 2024-04-26T20:27:09.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.373+0000 7f6e70a00640 1 -- 172.21.15.43:0/1729803338 wait complete. 2024-04-26T20:27:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.375+0000 7f6e70a00640 1 Processor -- start 2024-04-26T20:27:09.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.375+0000 7f6e70a00640 1 -- start start 2024-04-26T20:27:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c0b5410 0x7f6e6c140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15d9c0 0x7f6e6c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0b4b90 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c15e010 con 0x7f6e6c0b5410 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e70a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c15e2e0 con 0x7f6e6c15d9c0 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e69e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e69e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45258/0 (socket says 172.21.15.43:45258) 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.376+0000 7f6e69e00640 1 -- 172.21.15.43:0/594247383 learned_addr learned my addr 172.21.15.43:0/594247383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e6a800640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c0b5410 0x7f6e6c140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e6b200640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15d9c0 0x7f6e6c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 -- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15d9c0 msgr2=0x7f6e6c152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15d9c0 0x7f6e6c152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 -- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c0b5410 msgr2=0x7f6e6c140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c0b5410 0x7f6e6c140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 -- 172.21.15.43:0/594247383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1f2960 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e69e00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c460 tx=0x7f6e5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.377+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58011790 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.378+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e58017070 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.378+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58011bb0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.378+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c141700 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.378+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c141cc0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.379+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c0506c0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.380+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e5801c460 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.381+0000 7f6e62a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.381+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f6e5808bde0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.382+0000 7f6e6a800640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.382+0000 7f6e6a800640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.386+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e58059de0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.592+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6e6c0b4e30 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.593+0000 7f6e62a00640 1 -- 172.21.15.43:0/594247383 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6e580596e0 con 0x7f6e6c15a7e0 2024-04-26T20:27:09.594 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 msgr2=0x7f6e4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c008040 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 msgr2=0x7f6e6c0b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800c460 tx=0x7f6e5800c920 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 shutdown_connections 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e6c15d9c0 0x7f6e6c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e6c15a7e0 0x7f6e6c0b1ec0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 --2- 172.21.15.43:0/594247383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e6c0b5410 0x7f6e6c140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 >> 172.21.15.43:0/594247383 conn(0x7f6e6c068090 msgr2=0x7f6e6c15b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 shutdown_connections 2024-04-26T20:27:09.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.596+0000 7f6e70a00640 1 -- 172.21.15.43:0/594247383 wait complete. 2024-04-26T20:27:09.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.815+0000 7f45ffe00640 1 Processor -- start 2024-04-26T20:27:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.815+0000 7f45ffe00640 1 -- start start 2024-04-26T20:27:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.816+0000 7f45ffe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45f815f620 0x7f45f80b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.816+0000 7f45ffe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f815fbe0 con 0x7f45f806ca60 2024-04-26T20:27:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45ffe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f815fec0 con 0x7f45f8148950 2024-04-26T20:27:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45ffe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f45f80b51b0 con 0x7f45f815f620 2024-04-26T20:27:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fde00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45f815f620 0x7f45f80b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fca00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45f806ca60 0x7f45f80b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51894/0 (socket says 172.21.15.43:51894) 2024-04-26T20:27:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fca00640 1 -- 172.21.15.43:0/2989171101 learned_addr learned my addr 172.21.15.43:0/2989171101 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fde00640 1 -- 172.21.15.43:0/2989171101 >> v1:172.21.15.164:6789/0 conn(0x7f45f8148950 legacy=0x7f45f806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fde00640 1 -- 172.21.15.43:0/2989171101 >> v1:172.21.15.43:6789/0 conn(0x7f45f806ca60 legacy=0x7f45f80b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45f7e00640 1 -- 172.21.15.43:0/2989171101 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256991594 0 0) 0x7f45f815fbe0 con 0x7f45f806ca60 2024-04-26T20:27:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.817+0000 7f45fde00640 1 -- 172.21.15.43:0/2989171101 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f81601a0 con 0x7f45f815f620 2024-04-26T20:27:09.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45fde00640 1 --2- 172.21.15.43:0/2989171101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f815f620 0x7f45f80b0e60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45f00097a0 tx=0x7f45f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dad930126670580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45f7e00640 1 -- 172.21.15.43:0/2989171101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0037070 con 0x7f45f815f620 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45f7e00640 1 -- 172.21.15.43:0/2989171101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f0032440 con 0x7f45f815f620 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45f7e00640 1 -- 172.21.15.43:0/2989171101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0032810 con 0x7f45f815f620 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45ffe00640 1 -- 172.21.15.43:0/2989171101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f815f620 msgr2=0x7f45f80b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.819+0000 7f45ffe00640 1 --2- 172.21.15.43:0/2989171101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f815f620 0x7f45f80b0e60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45f00097a0 tx=0x7f45f002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.820+0000 7f45ffe00640 1 -- 172.21.15.43:0/2989171101 shutdown_connections 2024-04-26T20:27:09.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.820+0000 7f45ffe00640 1 --2- 172.21.15.43:0/2989171101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f815f620 0x7f45f80b0e60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.820+0000 7f45ffe00640 1 -- 172.21.15.43:0/2989171101 >> 172.21.15.43:0/2989171101 conn(0x7f45f80b39b0 msgr2=0x7f45f80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.820+0000 7f45ffe00640 1 -- 172.21.15.43:0/2989171101 shutdown_connections 2024-04-26T20:27:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.820+0000 7f45ffe00640 1 -- 172.21.15.43:0/2989171101 wait complete. 2024-04-26T20:27:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.821+0000 7f45ffe00640 1 Processor -- start 2024-04-26T20:27:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.822+0000 7f45ffe00640 1 -- start start 2024-04-26T20:27:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.822+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.822+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f8148950 0x7f45f815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45ffe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f815f620 0x7f45f815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45ffe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80b4710 con 0x7f45f806ca60 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45ffe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45f8148d30 con 0x7f45f815f620 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45ffe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f45f80b4890 con 0x7f45f8148950 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45276/0 (socket says 172.21.15.43:45276) 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 -- 172.21.15.43:0/3292323025 learned_addr learned my addr 172.21.15.43:0/3292323025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 -- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f8148950 msgr2=0x7f45f815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f8148950 0x7f45f815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 -- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f815f620 msgr2=0x7f45f815dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f815f620 0x7f45f815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.823+0000 7f45fd400640 1 -- 172.21.15.43:0/3292323025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f8141570 con 0x7f45f806ca60 2024-04-26T20:27:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45fd400640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c7c0 tx=0x7f45e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8016020 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e800aa50 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8012420 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f8141850 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.824+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f8141e10 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.825+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f8069a60 con 0x7f45f806ca60 2024-04-26T20:27:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.826+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45e8019020 con 0x7f45f806ca60 2024-04-26T20:27:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.827+0000 7f45f5600640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 0x7f45d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:09.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.827+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f45e808bd30 con 0x7f45f806ca60 2024-04-26T20:27:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.828+0000 7f45fca00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 0x7f45d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:09.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.828+0000 7f45fca00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 0x7f45d806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45e4007fd0 tx=0x7f45e4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:09.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:09.833+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45e8020070 con 0x7f45f806ca60 2024-04-26T20:27:10.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.048+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45f80af290 con 0x7f45f806ca60 2024-04-26T20:27:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.048+0000 7f45f5600640 1 -- 172.21.15.43:0/3292323025 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f45e8059d30 con 0x7f45f806ca60 2024-04-26T20:27:10.049 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 msgr2=0x7f45d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 0x7f45d806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45e4007fd0 tx=0x7f45e4007920 comp rx=0 tx=0).stop 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 msgr2=0x7f45f8140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f45e800c7c0 tx=0x7f45e800cc80 comp rx=0 tx=0).stop 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 shutdown_connections 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45d806c8f0 0x7f45d806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f8148950 0x7f45f815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f815f620 0x7f45f815dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 --2- 172.21.15.43:0/3292323025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45f806ca60 0x7f45f8140f00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 >> 172.21.15.43:0/3292323025 conn(0x7f45f80b39b0 msgr2=0x7f45f806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 shutdown_connections 2024-04-26T20:27:10.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.051+0000 7f45ffe00640 1 -- 172.21.15.43:0/3292323025 wait complete. 2024-04-26T20:27:10.070 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:10.070 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:10.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:10.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.317+0000 7ff6b3200640 1 Processor -- start 2024-04-26T20:27:10.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.317+0000 7ff6b3200640 1 -- start start 2024-04-26T20:27:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.319+0000 7ff6b3200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6ac145330 0x7ff6ac0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.319+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac0afe30 0x7ff6ac162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.319+0000 7ff6b3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ac145710 con 0x7ff6ac1628e0 2024-04-26T20:27:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.319+0000 7ff6b3200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ac073dc0 con 0x7ff6ac145330 2024-04-26T20:27:10.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.319+0000 7ff6b3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac0ae960 con 0x7ff6ac0afe30 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6ac145330 0x7ff6ac0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6ac145330 0x7ff6ac0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45286/0 (socket says 172.21.15.43:45286) 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 -- 172.21.15.43:0/2228747064 learned_addr learned my addr 172.21.15.43:0/2228747064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b2800640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac0afe30 0x7ff6ac162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 -- 172.21.15.43:0/2228747064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac0afe30 msgr2=0x7ff6ac162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac0afe30 0x7ff6ac162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 -- 172.21.15.43:0/2228747064 >> v1:172.21.15.164:6789/0 conn(0x7ff6ac1628e0 legacy=0x7ff6ac0af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.320+0000 7ff6b1e00640 1 -- 172.21.15.43:0/2228747064 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac0aec40 con 0x7ff6ac145330 2024-04-26T20:27:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.322+0000 7ff6b1e00640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac145330 0x7ff6ac0b3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009870 tx=0x7ff6a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b3c3f3bb5ae5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.322+0000 7ff6b0a00640 1 -- 172.21.15.43:0/2228747064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0037070 con 0x7ff6ac145330 2024-04-26T20:27:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.322+0000 7ff6b0a00640 1 -- 172.21.15.43:0/2228747064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a0032440 con 0x7ff6ac145330 2024-04-26T20:27:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.322+0000 7ff6b0a00640 1 -- 172.21.15.43:0/2228747064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0032810 con 0x7ff6ac145330 2024-04-26T20:27:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 -- 172.21.15.43:0/2228747064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac145330 msgr2=0x7ff6ac0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac145330 0x7ff6ac0b3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009870 tx=0x7ff6a002a270 comp rx=0 tx=0).stop 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 -- 172.21.15.43:0/2228747064 shutdown_connections 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac0afe30 0x7ff6ac162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 --2- 172.21.15.43:0/2228747064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac145330 0x7ff6ac0b3940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 -- 172.21.15.43:0/2228747064 >> 172.21.15.43:0/2228747064 conn(0x7ff6ac068090 msgr2=0x7ff6ac068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 -- 172.21.15.43:0/2228747064 shutdown_connections 2024-04-26T20:27:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.323+0000 7ff6b3200640 1 -- 172.21.15.43:0/2228747064 wait complete. 2024-04-26T20:27:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.325+0000 7ff6b3200640 1 Processor -- start 2024-04-26T20:27:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.325+0000 7ff6b3200640 1 -- start start 2024-04-26T20:27:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6ac0afe30 0x7ff6ac15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac145330 0x7ff6ac141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0b3e80 con 0x7ff6ac1628e0 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0af200 con 0x7ff6ac0afe30 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac073000 con 0x7ff6ac145330 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45292/0 (socket says 172.21.15.43:45292) 2024-04-26T20:27:10.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.326+0000 7ff6b2800640 1 -- 172.21.15.43:0/4023833950 learned_addr learned my addr 172.21.15.43:0/4023833950 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 -- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac145330 msgr2=0x7ff6ac141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b1400640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac145330 0x7ff6ac141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b1e00640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6ac0afe30 0x7ff6ac15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac145330 0x7ff6ac141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 -- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6ac0afe30 msgr2=0x7ff6ac15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6ac0afe30 0x7ff6ac15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 -- 172.21.15.43:0/4023833950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac0cd460 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b2800640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007e70 tx=0x7ff6a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8014620 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a8019070 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8014a20 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.327+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac0cd740 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.328+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac0cdd00 con 0x7ff6ac1628e0 2024-04-26T20:27:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.329+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ac069350 con 0x7ff6ac1628e0 2024-04-26T20:27:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.330+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6a8014bc0 con 0x7ff6ac1628e0 2024-04-26T20:27:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.330+0000 7ff6a6000640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 0x7ff69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.331+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7ff6a808c8e0 con 0x7ff6ac1628e0 2024-04-26T20:27:10.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.331+0000 7ff6b1e00640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 0x7ff69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.332+0000 7ff6b1e00640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 0x7ff69006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.335+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6a8011260 con 0x7ff6ac1628e0 2024-04-26T20:27:10.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.541+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6ac0b4000 con 0x7ff6ac1628e0 2024-04-26T20:27:10.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.543+0000 7ff6a6000640 1 -- 172.21.15.43:0/4023833950 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7ff6a805a8d0 con 0x7ff6ac1628e0 2024-04-26T20:27:10.544 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:10.544 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 msgr2=0x7ff69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 0x7ff69006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 msgr2=0x7ff6ac141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8007e70 tx=0x7ff6a800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 shutdown_connections 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff69006c8f0 0x7ff69006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff6ac145330 0x7ff6ac141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ac1628e0 0x7ff6ac141c60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 --2- 172.21.15.43:0/4023833950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6ac0afe30 0x7ff6ac15b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.546+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 >> 172.21.15.43:0/4023833950 conn(0x7ff6ac068090 msgr2=0x7ff6ac15cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:10.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.547+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 shutdown_connections 2024-04-26T20:27:10.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.547+0000 7ff6b3200640 1 -- 172.21.15.43:0/4023833950 wait complete. 2024-04-26T20:27:10.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:10.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:10.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.772+0000 7f933d400640 1 Processor -- start 2024-04-26T20:27:10.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.772+0000 7f933d400640 1 -- start start 2024-04-26T20:27:10.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.774+0000 7f933d400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.774+0000 7f933d400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 0x7f9338162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.774+0000 7f933d400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933815f920 con 0x7f9338155cd0 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.774+0000 7f933d400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f933815fc00 con 0x7f93380b5940 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.774+0000 7f933d400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f933815fee0 con 0x7f933815f3f0 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.775+0000 7f932fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.775+0000 7f933ca00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 0x7f9338162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.775+0000 7f932fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:51760/0 (socket says 172.21.15.43:51760) 2024-04-26T20:27:10.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.775+0000 7f932fe00640 1 -- 172.21.15.43:0/1735317368 learned_addr learned my addr 172.21.15.43:0/1735317368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:10.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f932fe00640 1 -- 172.21.15.43:0/1735317368 >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 msgr2=0x7f9338162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:10.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f932fe00640 1 --2- 172.21.15.43:0/1735317368 >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 0x7f9338162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f932fe00640 1 -- 172.21.15.43:0/1735317368 >> v1:172.21.15.164:6789/0 conn(0x7f9338155cd0 legacy=0x7f9338072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f932fe00640 1 -- 172.21.15.43:0/1735317368 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93381601c0 con 0x7f933815f3f0 2024-04-26T20:27:10.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f9337400640 1 -- 172.21.15.43:0/1735317368 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800916761 0 0) 0x7f933815f920 con 0x7f9338155cd0 2024-04-26T20:27:10.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.776+0000 7f933ca00640 1 --2- 172.21.15.43:0/1735317368 >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 0x7f9338162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:10.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.777+0000 7f932fe00640 1 --2- 172.21.15.43:0/1735317368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9320009870 tx=0x7f932002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174bf8d49c4b0479 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.778+0000 7f9337400640 1 -- 172.21.15.43:0/1735317368 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320037070 con 0x7f933815f3f0 2024-04-26T20:27:10.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.778+0000 7f9337400640 1 -- 172.21.15.43:0/1735317368 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932002ae50 con 0x7f933815f3f0 2024-04-26T20:27:10.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.778+0000 7f9337400640 1 -- 172.21.15.43:0/1735317368 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320032860 con 0x7f933815f3f0 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.778+0000 7f933d400640 1 -- 172.21.15.43:0/1735317368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 msgr2=0x7f93380b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.778+0000 7f933d400640 1 --2- 172.21.15.43:0/1735317368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9320009870 tx=0x7f932002a300 comp rx=0 tx=0).stop 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 -- 172.21.15.43:0/1735317368 shutdown_connections 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 --2- 172.21.15.43:0/1735317368 >> v2:172.21.15.164:3300/0 conn(0x7f93380b5940 0x7f9338162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 --2- 172.21.15.43:0/1735317368 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f933815f3f0 0x7f93380b5400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 -- 172.21.15.43:0/1735317368 >> 172.21.15.43:0/1735317368 conn(0x7f9338068090 msgr2=0x7f93380684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:10.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 -- 172.21.15.43:0/1735317368 shutdown_connections 2024-04-26T20:27:10.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.779+0000 7f933d400640 1 -- 172.21.15.43:0/1735317368 wait complete. 2024-04-26T20:27:10.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.781+0000 7f933d400640 1 Processor -- start 2024-04-26T20:27:10.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.781+0000 7f933d400640 1 -- start start 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93380b5940 0x7f9338144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 0x7f93381f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93381560b0 con 0x7f9338155cd0 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9338072d10 con 0x7f933815f3f0 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.782+0000 7f933d400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9338156230 con 0x7f93380b5940 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45298/0 (socket says 172.21.15.43:45298) 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 -- 172.21.15.43:0/2839568266 learned_addr learned my addr 172.21.15.43:0/2839568266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f9337e00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93380b5940 0x7f9338144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f933ca00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 0x7f93381f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 -- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93380b5940 msgr2=0x7f9338144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93380b5940 0x7f9338144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 -- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 msgr2=0x7f93381f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 0x7f93381f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f932fe00640 1 -- 172.21.15.43:0/2839568266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9338161750 con 0x7f9338155cd0 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.783+0000 7f933ca00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 0x7f93381f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f932fe00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9320000c00 tx=0x7f93200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320041070 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932003c660 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320037040 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9338161a30 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.784+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9338161ff0 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.785+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93380506c0 con 0x7f9338155cd0 2024-04-26T20:27:10.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.786+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9320044070 con 0x7f9338155cd0 2024-04-26T20:27:10.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.787+0000 7f9334c00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 0x7f931406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:10.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.787+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f93200ac670 con 0x7f9338155cd0 2024-04-26T20:27:10.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.787+0000 7f9337e00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 0x7f931406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:10.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.788+0000 7f9337e00640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 0x7f931406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9328000f60 tx=0x7f9328009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:10.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.791+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f932007a820 con 0x7f9338155cd0 2024-04-26T20:27:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:10.999+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9338072630 con 0x7f9338155cd0 2024-04-26T20:27:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.000+0000 7f9334c00640 1 -- 172.21.15.43:0/2839568266 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f932007a120 con 0x7f9338155cd0 2024-04-26T20:27:11.000 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 msgr2=0x7f931406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 0x7f931406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9328000f60 tx=0x7f9328009040 comp rx=0 tx=0).stop 2024-04-26T20:27:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 msgr2=0x7f9338160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9320000c00 tx=0x7f93200046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 shutdown_connections 2024-04-26T20:27:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f931406c8f0 0x7f931406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f933815f3f0 0x7f93381f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9338155cd0 0x7f9338160fe0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 --2- 172.21.15.43:0/2839568266 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93380b5940 0x7f9338144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 >> 172.21.15.43:0/2839568266 conn(0x7f9338068090 msgr2=0x7f9338162e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.002+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 shutdown_connections 2024-04-26T20:27:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.003+0000 7f933d400640 1 -- 172.21.15.43:0/2839568266 wait complete. 2024-04-26T20:27:11.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.236+0000 7f64a1800640 1 Processor -- start 2024-04-26T20:27:11.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.236+0000 7f64a1800640 1 -- start start 2024-04-26T20:27:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a1800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 0x7f649c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a1800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f649c15d2b0 0x7f649c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a1800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c0b3900 con 0x7f649c15a7e0 2024-04-26T20:27:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a1800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f649c15abc0 con 0x7f649c15d2b0 2024-04-26T20:27:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a1800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f649c0b3b70 con 0x7f649c0b5410 2024-04-26T20:27:11.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a0e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f649c15d2b0 0x7f649c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a0e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f649c15d2b0 0x7f649c15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45306/0 (socket says 172.21.15.43:45306) 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.238+0000 7f64a0e00640 1 -- 172.21.15.43:0/3271222772 learned_addr learned my addr 172.21.15.43:0/3271222772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f6493e00640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 0x7f649c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f64a0e00640 1 -- 172.21.15.43:0/3271222772 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 msgr2=0x7f649c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f64a0e00640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 0x7f649c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f64a0e00640 1 -- 172.21.15.43:0/3271222772 >> v1:172.21.15.164:6789/0 conn(0x7f649c15a7e0 legacy=0x7f649c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f64a0e00640 1 -- 172.21.15.43:0/3271222772 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c0b3e50 con 0x7f649c15d2b0 2024-04-26T20:27:11.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.239+0000 7f6493e00640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 0x7f649c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.240+0000 7f64a0e00640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15d2b0 0x7f649c15d740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6494009e50 tx=0x7f649402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841131507936c5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.240+0000 7f649b400640 1 -- 172.21.15.43:0/3271222772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494036070 con 0x7f649c15d2b0 2024-04-26T20:27:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.240+0000 7f649b400640 1 -- 172.21.15.43:0/3271222772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494031600 con 0x7f649c15d2b0 2024-04-26T20:27:11.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.240+0000 7f649b400640 1 -- 172.21.15.43:0/3271222772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940319d0 con 0x7f649c15d2b0 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 -- 172.21.15.43:0/3271222772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15d2b0 msgr2=0x7f649c15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15d2b0 0x7f649c15d740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6494009e50 tx=0x7f649402a710 comp rx=0 tx=0).stop 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 -- 172.21.15.43:0/3271222772 shutdown_connections 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15d2b0 0x7f649c15d740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 --2- 172.21.15.43:0/3271222772 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c0b5410 0x7f649c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 -- 172.21.15.43:0/3271222772 >> 172.21.15.43:0/3271222772 conn(0x7f649c068090 msgr2=0x7f649c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 -- 172.21.15.43:0/3271222772 shutdown_connections 2024-04-26T20:27:11.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.241+0000 7f64a1800640 1 -- 172.21.15.43:0/3271222772 wait complete. 2024-04-26T20:27:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.243+0000 7f64a1800640 1 Processor -- start 2024-04-26T20:27:11.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.243+0000 7f64a1800640 1 -- start start 2024-04-26T20:27:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f649c0b5410 0x7f649c0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 0x7f649c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f649c15dc80 con 0x7f649c15a7e0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f649c15e010 con 0x7f649c0b5410 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f64a1800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f649c15e2e0 con 0x7f649c15d2b0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f6493e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f6493e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45310/0 (socket says 172.21.15.43:45310) 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.244+0000 7f6493e00640 1 -- 172.21.15.43:0/1006842659 learned_addr learned my addr 172.21.15.43:0/1006842659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f649be00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f649c0b5410 0x7f649c0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f64a0e00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 0x7f649c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 -- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 msgr2=0x7f649c0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 0x7f649c0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 -- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f649c0b5410 msgr2=0x7f649c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f649c0b5410 0x7f649c0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 -- 172.21.15.43:0/1006842659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c0b0b40 con 0x7f649c15a7e0 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f64a0e00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 0x7f649c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:11.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6493e00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f648400e730 tx=0x7f648400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.245+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648400f040 con 0x7f649c15a7e0 2024-04-26T20:27:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.246+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6484004230 con 0x7f649c15a7e0 2024-04-26T20:27:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.246+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648400ee40 con 0x7f649c15a7e0 2024-04-26T20:27:11.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.246+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c0b0e20 con 0x7f649c15a7e0 2024-04-26T20:27:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.246+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c0b13e0 con 0x7f649c15a7e0 2024-04-26T20:27:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.247+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c0506c0 con 0x7f649c15a7e0 2024-04-26T20:27:11.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.248+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f648400c6d0 con 0x7f649c15a7e0 2024-04-26T20:27:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.249+0000 7f6498c00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 0x7f647806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.249+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f648408bca0 con 0x7f649c15a7e0 2024-04-26T20:27:11.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.249+0000 7f649be00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 0x7f647806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.250+0000 7f649be00640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 0x7f647806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f648c001640 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.253+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6484017020 con 0x7f649c15a7e0 2024-04-26T20:27:11.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.461+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f649c06f340 con 0x7f649c15a7e0 2024-04-26T20:27:11.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.462+0000 7f6498c00640 1 -- 172.21.15.43:0/1006842659 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f6484059d30 con 0x7f649c15a7e0 2024-04-26T20:27:11.463 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.464+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 msgr2=0x7f647806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.464+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 0x7f647806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f648c001640 tx=0x7f648c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 msgr2=0x7f649c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f648400e730 tx=0x7f648400ebf0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 shutdown_connections 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f647806c8f0 0x7f647806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f649c15d2b0 0x7f649c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15a7e0 0x7f649c0b0460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 --2- 172.21.15.43:0/1006842659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f649c0b5410 0x7f649c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 >> 172.21.15.43:0/1006842659 conn(0x7f649c068090 msgr2=0x7f649c158320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:11.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 shutdown_connections 2024-04-26T20:27:11.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.465+0000 7f64a1800640 1 -- 172.21.15.43:0/1006842659 wait complete. 2024-04-26T20:27:11.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:11.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.700+0000 7f8f8f600640 1 Processor -- start 2024-04-26T20:27:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.700+0000 7f8f8f600640 1 -- start start 2024-04-26T20:27:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8f600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f88145d80 0x7f8f88072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8f600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8815d5e0 0x7f8f880728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8f600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f88146160 con 0x7f8f881468c0 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8f600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8f8806dfb0 con 0x7f8f8815d5e0 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8f600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880d0470 con 0x7f8f88145d80 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8d600640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8f881468c0 0x7f8f8806ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51900/0 (socket says 172.21.15.43:51900) 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f8d600640 1 -- 172.21.15.43:0/2621788554 learned_addr learned my addr 172.21.15.43:0/2621788554 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.702+0000 7f8f87e00640 1 --2- 172.21.15.43:0/2621788554 >> v2:172.21.15.43:3300/0 conn(0x7f8f8815d5e0 0x7f8f880728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f8cc00640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f88145d80 0x7f8f88072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f87e00640 1 -- 172.21.15.43:0/2621788554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f88145d80 msgr2=0x7f8f88072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f87e00640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f88145d80 0x7f8f88072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f87e00640 1 -- 172.21.15.43:0/2621788554 >> v1:172.21.15.43:6789/0 conn(0x7f8f881468c0 legacy=0x7f8f8806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f87e00640 1 -- 172.21.15.43:0/2621788554 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f88074180 con 0x7f8f8815d5e0 2024-04-26T20:27:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.703+0000 7f8f87400640 1 -- 172.21.15.43:0/2621788554 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705484041 0 0) 0x7f8f88146160 con 0x7f8f881468c0 2024-04-26T20:27:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.704+0000 7f8f87e00640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f880728c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f740097a0 tx=0x7f8f7402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab549775cff7a565 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.704+0000 7f8f87400640 1 -- 172.21.15.43:0/2621788554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74037070 con 0x7f8f8815d5e0 2024-04-26T20:27:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.704+0000 7f8f87400640 1 -- 172.21.15.43:0/2621788554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7402ae50 con 0x7f8f8815d5e0 2024-04-26T20:27:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.704+0000 7f8f87400640 1 -- 172.21.15.43:0/2621788554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74032660 con 0x7f8f8815d5e0 2024-04-26T20:27:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 -- 172.21.15.43:0/2621788554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 msgr2=0x7f8f880728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f880728c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f740097a0 tx=0x7f8f7402a100 comp rx=0 tx=0).stop 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 -- 172.21.15.43:0/2621788554 shutdown_connections 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f880728c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 --2- 172.21.15.43:0/2621788554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f88145d80 0x7f8f88072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 -- 172.21.15.43:0/2621788554 >> 172.21.15.43:0/2621788554 conn(0x7f8f880b2900 msgr2=0x7f8f880b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 -- 172.21.15.43:0/2621788554 shutdown_connections 2024-04-26T20:27:11.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.705+0000 7f8f8f600640 1 -- 172.21.15.43:0/2621788554 wait complete. 2024-04-26T20:27:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.707+0000 7f8f8f600640 1 Processor -- start 2024-04-26T20:27:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.707+0000 7f8f8f600640 1 -- start start 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f88145d80 0x7f8f881f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f881468c0 0x7f8f881f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8806e530 con 0x7f8f8815d5e0 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8806e250 con 0x7f8f88145d80 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8f600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880d09f0 con 0x7f8f881468c0 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45328/0 (socket says 172.21.15.43:45328) 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.708+0000 7f8f8d600640 1 -- 172.21.15.43:0/3352672145 learned_addr learned my addr 172.21.15.43:0/3352672145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f87e00640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f881468c0 0x7f8f881f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8cc00640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f88145d80 0x7f8f881f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 -- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f881468c0 msgr2=0x7f8f881f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f881468c0 0x7f8f881f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 -- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f88145d80 msgr2=0x7f8f881f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f88145d80 0x7f8f881f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 -- 172.21.15.43:0/3352672145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8813a550 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8d600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c730 tx=0x7f8f8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80018070 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8000ce40 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000a860 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.709+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8813a830 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.710+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881f3f20 con 0x7f8f8815d5e0 2024-04-26T20:27:11.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.711+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f88069350 con 0x7f8f8815d5e0 2024-04-26T20:27:11.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.712+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f8001b070 con 0x7f8f8815d5e0 2024-04-26T20:27:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.713+0000 7f8f84c00640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 0x7f8f6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.713+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f8f8008b610 con 0x7f8f8815d5e0 2024-04-26T20:27:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.713+0000 7f8f8cc00640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 0x7f8f6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:11.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.714+0000 7f8f8cc00640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 0x7f8f6806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c006fd0 tx=0x7f8f7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:11.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.718+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f80059570 con 0x7f8f8815d5e0 2024-04-26T20:27:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.922+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f880b1720 con 0x7f8f8815d5e0 2024-04-26T20:27:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.923+0000 7f8f84c00640 1 -- 172.21.15.43:0/3352672145 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f8f80058e70 con 0x7f8f8815d5e0 2024-04-26T20:27:11.923 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 msgr2=0x7f8f6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 0x7f8f6806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c006fd0 tx=0x7f8f7c008040 comp rx=0 tx=0).stop 2024-04-26T20:27:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 msgr2=0x7f8f88142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c730 tx=0x7f8f8000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 shutdown_connections 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8f6806c920 0x7f8f6806ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f881468c0 0x7f8f881f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8815d5e0 0x7f8f88142700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 --2- 172.21.15.43:0/3352672145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f88145d80 0x7f8f881f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 >> 172.21.15.43:0/3352672145 conn(0x7f8f880b2900 msgr2=0x7f8f880d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 shutdown_connections 2024-04-26T20:27:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:11.926+0000 7f8f8f600640 1 -- 172.21.15.43:0/3352672145 wait complete. 2024-04-26T20:27:11.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:12.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.160+0000 7fb41d800640 1 Processor -- start 2024-04-26T20:27:12.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.160+0000 7fb41d800640 1 -- start start 2024-04-26T20:27:12.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb41d800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4180cf440 0x7fb4180b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb41d800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb41815d5f0 0x7fb41815da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb41d800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41806eaf0 con 0x7fb4180cf820 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb41d800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb41806f420 con 0x7fb41815d5f0 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb41d800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb41806edd0 con 0x7fb4180cf440 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb417400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb41815d5f0 0x7fb41815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb40fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb4180cf820 0x7fb4180cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51908/0 (socket says 172.21.15.43:51908) 2024-04-26T20:27:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.162+0000 7fb40fe00640 1 -- 172.21.15.43:0/2757867583 learned_addr learned my addr 172.21.15.43:0/2757867583 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb416a00640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4180cf440 0x7fb4180b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb417400640 1 -- 172.21.15.43:0/2757867583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4180cf440 msgr2=0x7fb4180b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb417400640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4180cf440 0x7fb4180b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb417400640 1 -- 172.21.15.43:0/2757867583 >> v1:172.21.15.43:6789/0 conn(0x7fb4180cf820 legacy=0x7fb4180cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb417400640 1 -- 172.21.15.43:0/2757867583 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41806f0b0 con 0x7fb41815d5f0 2024-04-26T20:27:12.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.163+0000 7fb416000640 1 -- 172.21.15.43:0/2757867583 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2750603365 0 0) 0x7fb41806eaf0 con 0x7fb4180cf820 2024-04-26T20:27:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.164+0000 7fb417400640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb41815d5f0 0x7fb41815da80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb410009d90 tx=0x7fb41002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182755389bfae642 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.164+0000 7fb416000640 1 -- 172.21.15.43:0/2757867583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410031600 con 0x7fb41815d5f0 2024-04-26T20:27:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.164+0000 7fb416000640 1 -- 172.21.15.43:0/2757867583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4100317a0 con 0x7fb41815d5f0 2024-04-26T20:27:12.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.164+0000 7fb416000640 1 -- 172.21.15.43:0/2757867583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410031b70 con 0x7fb41815d5f0 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 -- 172.21.15.43:0/2757867583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb41815d5f0 msgr2=0x7fb41815da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb41815d5f0 0x7fb41815da80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb410009d90 tx=0x7fb41002a5d0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 -- 172.21.15.43:0/2757867583 shutdown_connections 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb41815d5f0 0x7fb41815da80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 --2- 172.21.15.43:0/2757867583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4180cf440 0x7fb4180b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 -- 172.21.15.43:0/2757867583 >> 172.21.15.43:0/2757867583 conn(0x7fb4180b2900 msgr2=0x7fb4180b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 -- 172.21.15.43:0/2757867583 shutdown_connections 2024-04-26T20:27:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.165+0000 7fb41d800640 1 -- 172.21.15.43:0/2757867583 wait complete. 2024-04-26T20:27:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.167+0000 7fb41d800640 1 Processor -- start 2024-04-26T20:27:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.167+0000 7fb41d800640 1 -- start start 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180cf820 0x7fb41813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb41815d5f0 0x7fb41813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb4180d0390 con 0x7fb4180cf440 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb41806f8b0 con 0x7fb4180cf820 2024-04-26T20:27:12.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb41d800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb4180d0510 con 0x7fb41815d5f0 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb416a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb416a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45350/0 (socket says 172.21.15.43:45350) 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.168+0000 7fb416a00640 1 -- 172.21.15.43:0/193962034 learned_addr learned my addr 172.21.15.43:0/193962034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb40fe00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180cf820 0x7fb41813f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb417400640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb41815d5f0 0x7fb41813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 -- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb41815d5f0 msgr2=0x7fb41813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb41815d5f0 0x7fb41813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 -- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180cf820 msgr2=0x7fb41813f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180cf820 0x7fb41813f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 -- 172.21.15.43:0/193962034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41815e830 con 0x7fb4180cf440 2024-04-26T20:27:12.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb416a00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb40800d620 tx=0x7fb40800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800b620 con 0x7fb4180cf440 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb408019070 con 0x7fb4180cf440 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800ba00 con 0x7fb4180cf440 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.169+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41815eb10 con 0x7fb4180cf440 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.170+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41815f0d0 con 0x7fb4180cf440 2024-04-26T20:27:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.171+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418069350 con 0x7fb4180cf440 2024-04-26T20:27:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.172+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb40800bba0 con 0x7fb4180cf440 2024-04-26T20:27:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.173+0000 7fb40ea00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 0x7fb3f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.173+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fb40808c420 con 0x7fb4180cf440 2024-04-26T20:27:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.173+0000 7fb40fe00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 0x7fb3f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.174+0000 7fb40fe00640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 0x7fb3f406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4181436e0 tx=0x7fb400007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.178+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb40805a380 con 0x7fb4180cf440 2024-04-26T20:27:12.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.383+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb418145e90 con 0x7fb4180cf440 2024-04-26T20:27:12.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.385+0000 7fb40ea00640 1 -- 172.21.15.43:0/193962034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fb408059c80 con 0x7fb4180cf440 2024-04-26T20:27:12.385 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 msgr2=0x7fb3f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 0x7fb3f406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4181436e0 tx=0x7fb400007c00 comp rx=0 tx=0).stop 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 msgr2=0x7fb41815a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb40800d620 tx=0x7fb40800dae0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 shutdown_connections 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fb3f406c8f0 0x7fb3f406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb41815d5f0 0x7fb41813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4180cf820 0x7fb41813f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 --2- 172.21.15.43:0/193962034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4180cf440 0x7fb41815a3e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.387+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 >> 172.21.15.43:0/193962034 conn(0x7fb4180b2900 msgr2=0x7fb4180cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:12.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.388+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 shutdown_connections 2024-04-26T20:27:12.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.388+0000 7fb41d800640 1 -- 172.21.15.43:0/193962034 wait complete. 2024-04-26T20:27:12.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.625+0000 7f8125000640 1 Processor -- start 2024-04-26T20:27:12.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.625+0000 7f8125000640 1 -- start start 2024-04-26T20:27:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.627+0000 7f8125000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f8120072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.627+0000 7f8125000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f81200b3ce0 0x7f81200d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.627+0000 7f8125000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8120146ba0 con 0x7f81200b4140 2024-04-26T20:27:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.627+0000 7f8125000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8120073dd0 con 0x7f81200b3ce0 2024-04-26T20:27:12.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.627+0000 7f8125000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f81200731c0 con 0x7f81201467c0 2024-04-26T20:27:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f811e800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f81200b4140 0x7f8120160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:57334/0 (socket says 172.21.15.43:57334) 2024-04-26T20:27:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f811de00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f8120072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f811e800640 1 -- 172.21.15.43:0/3770831382 learned_addr learned my addr 172.21.15.43:0/3770831382 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:12.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f8117e00640 1 --2- 172.21.15.43:0/3770831382 >> v2:172.21.15.164:3300/0 conn(0x7f81200b3ce0 0x7f81200d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f8117e00640 1 -- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 msgr2=0x7f8120072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f8117e00640 1 --2- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f8120072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f8117e00640 1 -- 172.21.15.43:0/3770831382 >> v1:172.21.15.164:6789/0 conn(0x7f81200b4140 legacy=0x7f8120160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.628+0000 7f8117e00640 1 -- 172.21.15.43:0/3770831382 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81200734a0 con 0x7f81200b3ce0 2024-04-26T20:27:12.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.630+0000 7f8117e00640 1 --2- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200d0b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8120073f50 tx=0x7f810802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744b7d363ee6d4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.630+0000 7f811d400640 1 -- 172.21.15.43:0/3770831382 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108037070 con 0x7f81200b3ce0 2024-04-26T20:27:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.630+0000 7f811d400640 1 -- 172.21.15.43:0/3770831382 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108032440 con 0x7f81200b3ce0 2024-04-26T20:27:12.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.630+0000 7f811d400640 1 -- 172.21.15.43:0/3770831382 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108032810 con 0x7f81200b3ce0 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 -- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 msgr2=0x7f81200d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 --2- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200d0b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8120073f50 tx=0x7f810802a180 comp rx=0 tx=0).stop 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 -- 172.21.15.43:0/3770831382 shutdown_connections 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 --2- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200d0b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 --2- 172.21.15.43:0/3770831382 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f8120072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.631+0000 7f8125000640 1 -- 172.21.15.43:0/3770831382 >> 172.21.15.43:0/3770831382 conn(0x7f81200b2900 msgr2=0x7f81200b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.632+0000 7f8125000640 1 -- 172.21.15.43:0/3770831382 shutdown_connections 2024-04-26T20:27:12.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.632+0000 7f8125000640 1 -- 172.21.15.43:0/3770831382 wait complete. 2024-04-26T20:27:12.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.634+0000 7f8125000640 1 Processor -- start 2024-04-26T20:27:12.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.634+0000 7f8125000640 1 -- start start 2024-04-26T20:27:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f812013e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f812006f000 con 0x7f81200afec0 2024-04-26T20:27:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8120073a60 con 0x7f81200b3ce0 2024-04-26T20:27:12.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.635+0000 7f8125000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f812006f180 con 0x7f81201467c0 2024-04-26T20:27:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45364/0 (socket says 172.21.15.43:45364) 2024-04-26T20:27:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 -- 172.21.15.43:0/155556100 learned_addr learned my addr 172.21.15.43:0/155556100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811de00640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f8117e00640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f812013e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 -- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 msgr2=0x7f812013e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f812013e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 -- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 msgr2=0x7f81200ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.636+0000 7f811e800640 1 -- 172.21.15.43:0/155556100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81200aeea0 con 0x7f81200afec0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f811e800640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f811800cb20 tx=0x7f811800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118017070 con 0x7f81200afec0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811800aca0 con 0x7f81200afec0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118012600 con 0x7f81200afec0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81200af180 con 0x7f81200afec0 2024-04-26T20:27:12.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.637+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81200af740 con 0x7f81200afec0 2024-04-26T20:27:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.638+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8120069350 con 0x7f81200afec0 2024-04-26T20:27:12.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.639+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f811801a070 con 0x7f81200afec0 2024-04-26T20:27:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.641+0000 7f8116000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.641+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f811808bca0 con 0x7f81200afec0 2024-04-26T20:27:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.641+0000 7f811de00640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:12.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.642+0000 7f811de00640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 0x7f80fc06ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8110009aa0 tx=0x7f8110009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.646+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81180595b0 con 0x7f81200afec0 2024-04-26T20:27:12.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.864+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8120143710 con 0x7f81200afec0 2024-04-26T20:27:12.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.864+0000 7f8116000640 1 -- 172.21.15.43:0/155556100 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f811805d8b0 con 0x7f81200afec0 2024-04-26T20:27:12.865 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:12.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 msgr2=0x7f80fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 0x7f80fc06ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8110009aa0 tx=0x7f8110009190 comp rx=0 tx=0).stop 2024-04-26T20:27:12.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 msgr2=0x7f81200cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f811800cb20 tx=0x7f811800a060 comp rx=0 tx=0).stop 2024-04-26T20:27:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 shutdown_connections 2024-04-26T20:27:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f81201467c0 0x7f812013e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81200afec0 0x7f81200cd5a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 --2- 172.21.15.43:0/155556100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81200b3ce0 0x7f81200ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.867+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 >> 172.21.15.43:0/155556100 conn(0x7f81200b2900 msgr2=0x7f81200b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.868+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 shutdown_connections 2024-04-26T20:27:12.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:12.868+0000 7f8125000640 1 -- 172.21.15.43:0/155556100 wait complete. 2024-04-26T20:27:12.888 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-26T20:27:12.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:12.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:27:12.964 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:12.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:27:12.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:12.965 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:27:13.018 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:13.018 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:13.076 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:13.076 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:13.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.347+0000 7f91e7800640 1 Processor -- start 2024-04-26T20:27:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.348+0000 7f91e7800640 1 -- start start 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.349+0000 7f91e7800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f91e006f730 0x7f91e0166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e7800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e00cf060 con 0x7f91e0145590 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e7800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e00cf340 con 0x7f91e00b3490 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e7800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f91e00d0100 con 0x7f91e006f730 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e4e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91e0145590 0x7f91e00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51912/0 (socket says 172.21.15.43:51912) 2024-04-26T20:27:13.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e4e00640 1 -- 172.21.15.43:0/1126339974 learned_addr learned my addr 172.21.15.43:0/1126339974 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.350+0000 7f91e5800640 1 --2- 172.21.15.43:0/1126339974 >> v2:172.21.15.43:3300/0 conn(0x7f91e006f730 0x7f91e0166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.351+0000 7f91e5800640 1 -- 172.21.15.43:0/1126339974 >> v1:172.21.15.164:6789/0 conn(0x7f91e00b3490 legacy=0x7f91e006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.351+0000 7f91e5800640 1 -- 172.21.15.43:0/1126339974 >> v1:172.21.15.43:6789/0 conn(0x7f91e0145590 legacy=0x7f91e00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.351+0000 7f91e5800640 1 -- 172.21.15.43:0/1126339974 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e00cf620 con 0x7f91e006f730 2024-04-26T20:27:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.351+0000 7f91dfe00640 1 -- 172.21.15.43:0/1126339974 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2153133575 0 0) 0x7f91e00cf060 con 0x7f91e0145590 2024-04-26T20:27:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.352+0000 7f91e5800640 1 --2- 172.21.15.43:0/1126339974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e0166950 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009d90 tx=0x7f91d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326bc8b5badddda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.352+0000 7f91dfe00640 1 -- 172.21.15.43:0/1126339974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8007740 con 0x7f91e006f730 2024-04-26T20:27:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.352+0000 7f91dfe00640 1 -- 172.21.15.43:0/1126339974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d80078e0 con 0x7f91e006f730 2024-04-26T20:27:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.352+0000 7f91dfe00640 1 -- 172.21.15.43:0/1126339974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8007cb0 con 0x7f91e006f730 2024-04-26T20:27:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 -- 172.21.15.43:0/1126339974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 msgr2=0x7f91e0166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 --2- 172.21.15.43:0/1126339974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e0166950 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009d90 tx=0x7f91d802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 -- 172.21.15.43:0/1126339974 shutdown_connections 2024-04-26T20:27:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 --2- 172.21.15.43:0/1126339974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e0166950 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 -- 172.21.15.43:0/1126339974 >> 172.21.15.43:0/1126339974 conn(0x7f91e00b2900 msgr2=0x7f91e00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 -- 172.21.15.43:0/1126339974 shutdown_connections 2024-04-26T20:27:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.353+0000 7f91e7800640 1 -- 172.21.15.43:0/1126339974 wait complete. 2024-04-26T20:27:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.355+0000 7f91e7800640 1 Processor -- start 2024-04-26T20:27:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.355+0000 7f91e7800640 1 -- start start 2024-04-26T20:27:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 0x7f91e01f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91e0145590 0x7f91e01f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0145970 con 0x7f91e006f730 2024-04-26T20:27:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91e00cfbe0 con 0x7f91e00b3490 2024-04-26T20:27:13.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e7800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f91e0072ec0 con 0x7f91e0145590 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e4e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45392/0 (socket says 172.21.15.43:45392) 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.356+0000 7f91e4e00640 1 -- 172.21.15.43:0/953922339 learned_addr learned my addr 172.21.15.43:0/953922339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91d7e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 0x7f91e01f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e5800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91e0145590 0x7f91e01f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 -- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91e0145590 msgr2=0x7f91e01f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91e0145590 0x7f91e01f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 -- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 msgr2=0x7f91e01f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 0x7f91e01f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 -- 172.21.15.43:0/953922339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e013f320 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91d7e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 0x7f91e01f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.357+0000 7f91e4e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c430 tx=0x7f91d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.358+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d00074e0 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.358+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d00047e0 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.358+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0004bd0 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.358+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e013f600 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.358+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e013fbc0 con 0x7f91e006f730 2024-04-26T20:27:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.359+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0069350 con 0x7f91e006f730 2024-04-26T20:27:13.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.360+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d0011070 con 0x7f91e006f730 2024-04-26T20:27:13.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.361+0000 7f91dd600640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.361+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f91d008bc50 con 0x7f91e006f730 2024-04-26T20:27:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.361+0000 7f91d7e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.362+0000 7f91d7e00640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 0x7f91bc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91e01e61b0 tx=0x7f91c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.366+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91d0059c40 con 0x7f91e006f730 2024-04-26T20:27:13.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.578+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91e0141c70 con 0x7f91e006f730 2024-04-26T20:27:13.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.581+0000 7f91dd600640 1 -- 172.21.15.43:0/953922339 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f91d0059540 con 0x7f91e006f730 2024-04-26T20:27:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:13.582 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:13.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 msgr2=0x7f91bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 0x7f91bc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91e01e61b0 tx=0x7f91c8009040 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 msgr2=0x7f91e01403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c430 tx=0x7f91d000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 shutdown_connections 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91e0145590 0x7f91e01f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91e00b3490 0x7f91e01f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 --2- 172.21.15.43:0/953922339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91e006f730 0x7f91e01403c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 >> 172.21.15.43:0/953922339 conn(0x7f91e00b2900 msgr2=0x7f91e00b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 shutdown_connections 2024-04-26T20:27:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.584+0000 7f91e7800640 1 -- 172.21.15.43:0/953922339 wait complete. 2024-04-26T20:27:13.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:13.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.809+0000 7f5263200640 1 Processor -- start 2024-04-26T20:27:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.809+0000 7f5263200640 1 -- start start 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.810+0000 7f5263200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f526415b980 0x7f526406ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5263200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f526415bd60 0x7f526406f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5263200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52640b4610 0x7f52640b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5263200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f526415d060 con 0x7f52640b4610 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5263200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f52640b4fe0 con 0x7f526415bd60 2024-04-26T20:27:13.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5263200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f526415c2a0 con 0x7f526415b980 2024-04-26T20:27:13.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5262800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52640b4610 0x7f52640b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5262800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52640b4610 0x7f52640b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45408/0 (socket says 172.21.15.43:45408) 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5262800640 1 -- 172.21.15.43:0/3822538059 learned_addr learned my addr 172.21.15.43:0/3822538059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5261e00640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f526415b980 0x7f526406ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.811+0000 7f5261400640 1 --2- 172.21.15.43:0/3822538059 >> v2:172.21.15.164:3300/0 conn(0x7f526415bd60 0x7f526406f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.812+0000 7f5262800640 1 -- 172.21.15.43:0/3822538059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f526415b980 msgr2=0x7f526406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.812+0000 7f5262800640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f526415b980 0x7f526406ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.812+0000 7f5262800640 1 -- 172.21.15.43:0/3822538059 >> v2:172.21.15.164:3300/0 conn(0x7f526415bd60 msgr2=0x7f526406f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.812+0000 7f5262800640 1 --2- 172.21.15.43:0/3822538059 >> v2:172.21.15.164:3300/0 conn(0x7f526415bd60 0x7f526406f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.812+0000 7f5262800640 1 -- 172.21.15.43:0/3822538059 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526415c580 con 0x7f52640b4610 2024-04-26T20:27:13.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.813+0000 7f5262800640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52640b4610 0x7f52640b4aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f52640d1870 tx=0x7f525c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249969401534d557 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.813+0000 7f5260a00640 1 -- 172.21.15.43:0/3822538059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00b2e0 con 0x7f52640b4610 2024-04-26T20:27:13.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.813+0000 7f5260a00640 1 -- 172.21.15.43:0/3822538059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525c009600 con 0x7f52640b4610 2024-04-26T20:27:13.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.813+0000 7f5260a00640 1 -- 172.21.15.43:0/3822538059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00b480 con 0x7f52640b4610 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 -- 172.21.15.43:0/3822538059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52640b4610 msgr2=0x7f52640b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52640b4610 0x7f52640b4aa0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f52640d1870 tx=0x7f525c02c8a0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 -- 172.21.15.43:0/3822538059 shutdown_connections 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52640b4610 0x7f52640b4aa0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 --2- 172.21.15.43:0/3822538059 >> v2:172.21.15.164:3300/0 conn(0x7f526415bd60 0x7f526406f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 --2- 172.21.15.43:0/3822538059 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f526415b980 0x7f526406ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 -- 172.21.15.43:0/3822538059 >> 172.21.15.43:0/3822538059 conn(0x7f5264068090 msgr2=0x7f52640684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 -- 172.21.15.43:0/3822538059 shutdown_connections 2024-04-26T20:27:13.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.814+0000 7f5263200640 1 -- 172.21.15.43:0/3822538059 wait complete. 2024-04-26T20:27:13.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.816+0000 7f5263200640 1 Processor -- start 2024-04-26T20:27:13.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.816+0000 7f5263200640 1 -- start start 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52640b4610 0x7f52640ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 0x7f5264149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52640b52b0 con 0x7f526415bd60 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f526415cb40 con 0x7f526415b980 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5263200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5264072cd0 con 0x7f52640b4610 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5262800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45416/0 (socket says 172.21.15.43:45416) 2024-04-26T20:27:13.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5262800640 1 -- 172.21.15.43:0/873287036 learned_addr learned my addr 172.21.15.43:0/873287036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5261e00640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52640b4610 0x7f52640ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5261400640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 0x7f5264149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5262800640 1 -- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52640b4610 msgr2=0x7f52640ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.818+0000 7f5262800640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52640b4610 0x7f52640ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5262800640 1 -- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 msgr2=0x7f5264149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5262800640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 0x7f5264149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5262800640 1 -- 172.21.15.43:0/873287036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5264156f30 con 0x7f526415bd60 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5261400640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 0x7f5264149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:13.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5262800640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f525c02ce90 tx=0x7f525c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c041070 con 0x7f526415bd60 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525c03d820 con 0x7f526415bd60 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c03dbd0 con 0x7f526415bd60 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.819+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52640aec70 con 0x7f526415bd60 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.820+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52640aa550 con 0x7f526415bd60 2024-04-26T20:27:13.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.821+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52640aa980 con 0x7f526415bd60 2024-04-26T20:27:13.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.822+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f525c0333f0 con 0x7f526415bd60 2024-04-26T20:27:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.823+0000 7f525a000640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 0x7f524406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.823+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f525c030020 con 0x7f526415bd60 2024-04-26T20:27:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.823+0000 7f5261e00640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 0x7f524406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.824+0000 7f5261e00640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 0x7f524406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:13.828+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f525c0b5080 con 0x7f526415bd60 2024-04-26T20:27:14.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.032+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f526415ad20 con 0x7f526415bd60 2024-04-26T20:27:14.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.033+0000 7f525a000640 1 -- 172.21.15.43:0/873287036 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5264156f30 con 0x7f526415bd60 2024-04-26T20:27:14.034 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 msgr2=0x7f524406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 0x7f524406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254008040 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 msgr2=0x7f5264149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f525c02ce90 tx=0x7f525c004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 shutdown_connections 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f524406c920 0x7f524406ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526415bd60 0x7f5264149650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f526415b980 0x7f5264149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 --2- 172.21.15.43:0/873287036 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f52640b4610 0x7f52640ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 >> 172.21.15.43:0/873287036 conn(0x7f5264068090 msgr2=0x7f526415ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 shutdown_connections 2024-04-26T20:27:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.036+0000 7f5263200640 1 -- 172.21.15.43:0/873287036 wait complete. 2024-04-26T20:27:14.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.263+0000 7f8582a00640 1 Processor -- start 2024-04-26T20:27:14.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.264+0000 7f8582a00640 1 -- start start 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.265+0000 7f8582a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f857c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8582a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f857c15bd60 0x7f857c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8582a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857c15c2a0 con 0x7f857c15b980 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8582a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f857c15d060 con 0x7f857c15bd60 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8582a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f857c0b5090 con 0x7f857c06f3a0 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8581600640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f857c15b980 0x7f857c06ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51928/0 (socket says 172.21.15.43:51928) 2024-04-26T20:27:14.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8581600640 1 -- 172.21.15.43:0/1696160355 learned_addr learned my addr 172.21.15.43:0/1696160355 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:14.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8580c00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f857c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387721124 0 0) 0x7f857c15c2a0 con 0x7f857c15b980 2024-04-26T20:27:14.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8564003520 con 0x7f857c15b980 2024-04-26T20:27:14.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.266+0000 7f8582000640 1 --2- 172.21.15.43:0/1696160355 >> v2:172.21.15.164:3300/0 conn(0x7f857c15bd60 0x7f857c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581359443 0 0) 0x7f8564003520 con 0x7f857c15b980 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 msgr2=0x7f857c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f857c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 >> v2:172.21.15.164:3300/0 conn(0x7f857c15bd60 msgr2=0x7f857c0b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 --2- 172.21.15.43:0/1696160355 >> v2:172.21.15.164:3300/0 conn(0x7f857c15bd60 0x7f857c0b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c15c580 con 0x7f857c15b980 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f85700019f0 con 0x7f857c15b980 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 >> v1:172.21.15.43:6789/0 conn(0x7f857c15b980 legacy=0x7f857c06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8564003a20 0x7f8564003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f8564004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f85640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f857c15d060 con 0x7f8564004d40 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b5090 con 0x7f8564003a20 2024-04-26T20:27:14.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f856400dc80 con 0x7f857c06f3a0 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.267+0000 7f8582000640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f85640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8580c00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f8564004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8581600640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8564003a20 0x7f8564003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8582000640 1 -- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 msgr2=0x7f8564004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8582000640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c06f3a0 0x7f8564004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8582000640 1 -- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8564003a20 msgr2=0x7f8564003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8582000640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8564003a20 0x7f8564003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.268+0000 7f8582000640 1 -- 172.21.15.43:0/1696160355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8564003520 con 0x7f8564004d40 2024-04-26T20:27:14.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.269+0000 7f8582000640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f85640051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f857400ba00 tx=0x7f8564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ad1bc3814280c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.269+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574030070 con 0x7f8564004d40 2024-04-26T20:27:14.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.269+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857402ade0 con 0x7f8564004d40 2024-04-26T20:27:14.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.269+0000 7f857be00640 1 -- 172.21.15.43:0/1696160355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400b600 con 0x7f8564004d40 2024-04-26T20:27:14.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 -- 172.21.15.43:0/1696160355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 msgr2=0x7f85640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 --2- 172.21.15.43:0/1696160355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f85640051c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f857400ba00 tx=0x7f8564005830 comp rx=0 tx=0).stop 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582000640 1 -- 172.21.15.43:0/1696160355 reap_dead start 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 -- 172.21.15.43:0/1696160355 shutdown_connections 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 -- 172.21.15.43:0/1696160355 >> 172.21.15.43:0/1696160355 conn(0x7f857c068090 msgr2=0x7f857c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 -- 172.21.15.43:0/1696160355 shutdown_connections 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.270+0000 7f8582a00640 1 -- 172.21.15.43:0/1696160355 wait complete. 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.272+0000 7f8582a00640 1 Processor -- start 2024-04-26T20:27:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.272+0000 7f8582a00640 1 -- start start 2024-04-26T20:27:14.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f857c0d0240 0x7f857c0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c0ce4f0 0x7f857c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b52b0 con 0x7f8564004d40 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f857c15cb40 con 0x7f857c0d0240 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8582a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f857c0b4610 con 0x7f857c0ce4f0 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45422/0 (socket says 172.21.15.43:45422) 2024-04-26T20:27:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 -- 172.21.15.43:0/574239214 learned_addr learned my addr 172.21.15.43:0/574239214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 -- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c0ce4f0 msgr2=0x7f857c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c0ce4f0 0x7f857c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 -- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f857c0d0240 msgr2=0x7f857c0d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f857c0d0240 0x7f857c0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.273+0000 7f8581600640 1 -- 172.21.15.43:0/574239214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c151960 con 0x7f8564004d40 2024-04-26T20:27:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8581600640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8570001350 tx=0x7f857000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570019070 con 0x7f8564004d40 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8570004d60 con 0x7f8564004d40 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000be60 con 0x7f8564004d40 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c0cf0d0 con 0x7f8564004d40 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.274+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c0c5900 con 0x7f8564004d40 2024-04-26T20:27:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.275+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c0506c0 con 0x7f8564004d40 2024-04-26T20:27:14.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.276+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f857001c070 con 0x7f8564004d40 2024-04-26T20:27:14.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.277+0000 7f8579600640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 0x7f855c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.278+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f857008bfa0 con 0x7f8564004d40 2024-04-26T20:27:14.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.278+0000 7f8580c00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 0x7f855c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.278+0000 7f8580c00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 0x7f855c06ee00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f857c0d1210 tx=0x7f8568008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.282+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85700599c0 con 0x7f8564004d40 2024-04-26T20:27:14.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.489+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f857c068540 con 0x7f8564004d40 2024-04-26T20:27:14.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.490+0000 7f8579600640 1 -- 172.21.15.43:0/574239214 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f857005dcc0 con 0x7f8564004d40 2024-04-26T20:27:14.491 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 msgr2=0x7f855c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 0x7f855c06ee00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f857c0d1210 tx=0x7f8568008040 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 msgr2=0x7f857c1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8570001350 tx=0x7f857000b4a0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 shutdown_connections 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f855c06c940 0x7f855c06ee00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f857c0ce4f0 0x7f857c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f857c0d0240 0x7f857c0d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 --2- 172.21.15.43:0/574239214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8564004d40 0x7f857c1556e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 >> 172.21.15.43:0/574239214 conn(0x7f857c068090 msgr2=0x7f857c15a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 shutdown_connections 2024-04-26T20:27:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.493+0000 7f8582a00640 1 -- 172.21.15.43:0/574239214 wait complete. 2024-04-26T20:27:14.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.730+0000 7f5c3aa00640 1 Processor -- start 2024-04-26T20:27:14.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.730+0000 7f5c3aa00640 1 -- start start 2024-04-26T20:27:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.731+0000 7f5c3aa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c341572b0 0x7f5c340b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.731+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c340d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c3aa00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3415d2e0 con 0x7f5c340af8a0 2024-04-26T20:27:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c3aa00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3415d5c0 con 0x7f5c341572b0 2024-04-26T20:27:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c3aa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3415d8a0 con 0x7f5c340d0bd0 2024-04-26T20:27:14.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c3a000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5c340af8a0 0x7f5c340afd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51932/0 (socket says 172.21.15.43:51932) 2024-04-26T20:27:14.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c3a000640 1 -- 172.21.15.43:0/804177422 learned_addr learned my addr 172.21.15.43:0/804177422 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:14.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c39600640 1 --2- 172.21.15.43:0/804177422 >> v2:172.21.15.43:3300/0 conn(0x7f5c341572b0 0x7f5c340b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.732+0000 7f5c38c00640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c340d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c33e00640 1 -- 172.21.15.43:0/804177422 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313491260 0 0) 0x7f5c3415d2e0 con 0x7f5c340af8a0 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c33e00640 1 -- 172.21.15.43:0/804177422 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c18003520 con 0x7f5c340af8a0 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c39600640 1 -- 172.21.15.43:0/804177422 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 msgr2=0x7f5c340d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c39600640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c340d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c39600640 1 -- 172.21.15.43:0/804177422 >> v1:172.21.15.43:6789/0 conn(0x7f5c340af8a0 legacy=0x7f5c340afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.733+0000 7f5c39600640 1 -- 172.21.15.43:0/804177422 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3415db80 con 0x7f5c341572b0 2024-04-26T20:27:14.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.734+0000 7f5c39600640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c340b3e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009d90 tx=0x7f5c2402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb534d80692633 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.734+0000 7f5c33e00640 1 -- 172.21.15.43:0/804177422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24007740 con 0x7f5c341572b0 2024-04-26T20:27:14.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.734+0000 7f5c33e00640 1 -- 172.21.15.43:0/804177422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c240078e0 con 0x7f5c341572b0 2024-04-26T20:27:14.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.734+0000 7f5c33e00640 1 -- 172.21.15.43:0/804177422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24007c90 con 0x7f5c341572b0 2024-04-26T20:27:14.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 -- 172.21.15.43:0/804177422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 msgr2=0x7f5c340b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c340b3e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009d90 tx=0x7f5c2402a530 comp rx=0 tx=0).stop 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 -- 172.21.15.43:0/804177422 shutdown_connections 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c340d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/804177422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c340b3e30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 -- 172.21.15.43:0/804177422 >> 172.21.15.43:0/804177422 conn(0x7f5c340684e0 msgr2=0x7f5c340688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 -- 172.21.15.43:0/804177422 shutdown_connections 2024-04-26T20:27:14.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.735+0000 7f5c3aa00640 1 -- 172.21.15.43:0/804177422 wait complete. 2024-04-26T20:27:14.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.737+0000 7f5c3aa00640 1 Processor -- start 2024-04-26T20:27:14.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.737+0000 7f5c3aa00640 1 -- start start 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c340af8a0 0x7f5c341590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c3414a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34160c00 con 0x7f5c341572b0 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34160a80 con 0x7f5c340af8a0 2024-04-26T20:27:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3aa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5c34160d80 con 0x7f5c340d0bd0 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45440/0 (socket says 172.21.15.43:45440) 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c3a000640 1 -- 172.21.15.43:0/3213125134 learned_addr learned my addr 172.21.15.43:0/3213125134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.738+0000 7f5c38c00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c3414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c39600640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c340af8a0 0x7f5c341590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 -- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 msgr2=0x7f5c3414a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c3414a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 -- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c340af8a0 msgr2=0x7f5c341590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c340af8a0 0x7f5c341590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 -- 172.21.15.43:0/3213125134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3414ac70 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3a000640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c430 tx=0x7f5c2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c017070 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c005ce0 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c34158000 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c007310 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.739+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c341585c0 con 0x7f5c341572b0 2024-04-26T20:27:14.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.740+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c340506c0 con 0x7f5c341572b0 2024-04-26T20:27:14.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.741+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c2c01a070 con 0x7f5c341572b0 2024-04-26T20:27:14.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.742+0000 7f5c31600640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 0x7f5c1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:14.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.743+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f5c2c08b3d0 con 0x7f5c341572b0 2024-04-26T20:27:14.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.743+0000 7f5c39600640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 0x7f5c1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:14.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.744+0000 7f5c39600640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 0x7f5c1406ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c24034000 tx=0x7f5c24033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:14.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.747+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c2c0593c0 con 0x7f5c341572b0 2024-04-26T20:27:14.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.953+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c34158cc0 con 0x7f5c341572b0 2024-04-26T20:27:14.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.954+0000 7f5c31600640 1 -- 172.21.15.43:0/3213125134 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5c2c058cc0 con 0x7f5c341572b0 2024-04-26T20:27:14.955 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 msgr2=0x7f5c1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 0x7f5c1406ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c24034000 tx=0x7f5c24033040 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 msgr2=0x7f5c34157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c430 tx=0x7f5c2c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 shutdown_connections 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c1406c920 0x7f5c1406ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c340d0bd0 0x7f5c3414a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c341572b0 0x7f5c34157690 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 --2- 172.21.15.43:0/3213125134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c340af8a0 0x7f5c341590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 >> 172.21.15.43:0/3213125134 conn(0x7f5c340684e0 msgr2=0x7f5c34074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 shutdown_connections 2024-04-26T20:27:14.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:14.957+0000 7f5c3aa00640 1 -- 172.21.15.43:0/3213125134 wait complete. 2024-04-26T20:27:14.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.183+0000 7f5c75400640 1 Processor -- start 2024-04-26T20:27:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.184+0000 7f5c75400640 1 -- start start 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c70160d10 0x7f5c700b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 0x7f5c7006e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006e550 0x7f5c70166d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5c70163800 con 0x7f5c70160d10 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7006e9a0 con 0x7f5c7006db80 2024-04-26T20:27:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c75400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5c70163a50 con 0x7f5c7006e550 2024-04-26T20:27:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c6fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c70160d10 0x7f5c700b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.186+0000 7f5c74a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006e550 0x7f5c70166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c70160d10 0x7f5c700b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45452/0 (socket says 172.21.15.43:45452) 2024-04-26T20:27:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 -- 172.21.15.43:0/2532780001 learned_addr learned my addr 172.21.15.43:0/2532780001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c67e00640 1 --2- 172.21.15.43:0/2532780001 >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 0x7f5c7006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 -- 172.21.15.43:0/2532780001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006e550 msgr2=0x7f5c70166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/2532780001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006e550 0x7f5c70166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 -- 172.21.15.43:0/2532780001 >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 msgr2=0x7f5c7006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/2532780001 >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 0x7f5c7006e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c6fe00640 1 -- 172.21.15.43:0/2532780001 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c70163d30 con 0x7f5c70160d10 2024-04-26T20:27:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.187+0000 7f5c67e00640 1 --2- 172.21.15.43:0/2532780001 >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 0x7f5c7006e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.188+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/2532780001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700b5600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b5f50 tx=0x7f5c6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9bf0a5c53b00a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.189+0000 7f5c6f400640 1 -- 172.21.15.43:0/2532780001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60037070 con 0x7f5c70160d10 2024-04-26T20:27:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.189+0000 7f5c6f400640 1 -- 172.21.15.43:0/2532780001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c60032440 con 0x7f5c70160d10 2024-04-26T20:27:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.189+0000 7f5c6f400640 1 -- 172.21.15.43:0/2532780001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60032810 con 0x7f5c70160d10 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.189+0000 7f5c75400640 1 -- 172.21.15.43:0/2532780001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 msgr2=0x7f5c700b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.189+0000 7f5c75400640 1 --2- 172.21.15.43:0/2532780001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700b5600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5c700b5f50 tx=0x7f5c6002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 -- 172.21.15.43:0/2532780001 shutdown_connections 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 --2- 172.21.15.43:0/2532780001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006e550 0x7f5c70166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 --2- 172.21.15.43:0/2532780001 >> v2:172.21.15.164:3300/0 conn(0x7f5c7006db80 0x7f5c7006e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 --2- 172.21.15.43:0/2532780001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700b5600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 -- 172.21.15.43:0/2532780001 >> 172.21.15.43:0/2532780001 conn(0x7f5c70068090 msgr2=0x7f5c700684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 -- 172.21.15.43:0/2532780001 shutdown_connections 2024-04-26T20:27:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.190+0000 7f5c75400640 1 -- 172.21.15.43:0/2532780001 wait complete. 2024-04-26T20:27:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.192+0000 7f5c75400640 1 Processor -- start 2024-04-26T20:27:15.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.192+0000 7f5c75400640 1 -- start start 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006db80 0x7f5c7014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c7006e550 0x7f5c700cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5c701610f0 con 0x7f5c70160d10 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5c701642f0 con 0x7f5c7006e550 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c75400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5c70164950 con 0x7f5c7006db80 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45458/0 (socket says 172.21.15.43:45458) 2024-04-26T20:27:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 -- 172.21.15.43:0/1660989436 learned_addr learned my addr 172.21.15.43:0/1660989436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c67e00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c7006e550 0x7f5c700cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006db80 0x7f5c7014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 -- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006db80 msgr2=0x7f5c7014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006db80 0x7f5c7014aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 -- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c7006e550 msgr2=0x7f5c700cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c7006e550 0x7f5c700cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.193+0000 7f5c74a00640 1 -- 172.21.15.43:0/1660989436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7015a120 con 0x7f5c70160d10 2024-04-26T20:27:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c74a00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800efc0 tx=0x7f5c6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68018020 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c68014480 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68014870 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c700cd710 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.194+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c700cdcd0 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.195+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c700cded0 con 0x7f5c70160d10 2024-04-26T20:27:15.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.196+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c68005d10 con 0x7f5c70160d10 2024-04-26T20:27:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.197+0000 7f5c6cc00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.197+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f5c6808ca60 con 0x7f5c70160d10 2024-04-26T20:27:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.198+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.199+0000 7f5c6fe00640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c60034000 tx=0x7f5c6002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.202+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c700cded0 con 0x7f5c70160d10 2024-04-26T20:27:15.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.403+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5c700506c0 con 0x7f5c70160d10 2024-04-26T20:27:15.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.405+0000 7f5c6cc00640 1 -- 172.21.15.43:0/1660989436 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f5c68022070 con 0x7f5c70160d10 2024-04-26T20:27:15.405 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:15.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 msgr2=0x7f5c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c60034000 tx=0x7f5c6002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 msgr2=0x7f5c700d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800efc0 tx=0x7f5c6800c490 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 shutdown_connections 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c70160d10 0x7f5c700d0260 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5c7006e550 0x7f5c700cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 --2- 172.21.15.43:0/1660989436 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5c7006db80 0x7f5c7014aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.407+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 >> 172.21.15.43:0/1660989436 conn(0x7f5c70068090 msgr2=0x7f5c700b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:15.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.408+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 shutdown_connections 2024-04-26T20:27:15.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.408+0000 7f5c75400640 1 -- 172.21.15.43:0/1660989436 wait complete. 2024-04-26T20:27:15.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.638+0000 7f96d3200640 1 Processor -- start 2024-04-26T20:27:15.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.638+0000 7f96d3200640 1 -- start start 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.639+0000 7f96d3200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96d415f170 0x7f96d40d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.639+0000 7f96d3200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f96d40d5850 0x7f96d40d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d40d0f10 con 0x7f96d40d36a0 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d3200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f96d40d0c30 con 0x7f96d415f170 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d3200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f96d40b42a0 con 0x7f96d40d5850 2024-04-26T20:27:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96d415f170 0x7f96d40d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96d415f170 0x7f96d40d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45468/0 (socket says 172.21.15.43:45468) 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 -- 172.21.15.43:0/3758406661 learned_addr learned my addr 172.21.15.43:0/3758406661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 -- 172.21.15.43:0/3758406661 >> v2:172.21.15.164:3300/0 conn(0x7f96d40d5850 msgr2=0x7f96d40d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 --2- 172.21.15.43:0/3758406661 >> v2:172.21.15.164:3300/0 conn(0x7f96d40d5850 0x7f96d40d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 -- 172.21.15.43:0/3758406661 >> v1:172.21.15.164:6789/0 conn(0x7f96d40d36a0 legacy=0x7f96d40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:15.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.640+0000 7f96d1e00640 1 -- 172.21.15.43:0/3758406661 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d40b4580 con 0x7f96d415f170 2024-04-26T20:27:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.641+0000 7f96d1e00640 1 --2- 172.21.15.43:0/3758406661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d415f170 0x7f96d40d2a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d408d2e0 tx=0x7f96c402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6c74a8d70ce4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d0a00640 1 -- 172.21.15.43:0/3758406661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4036070 con 0x7f96d415f170 2024-04-26T20:27:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d0a00640 1 -- 172.21.15.43:0/3758406661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c4031600 con 0x7f96d415f170 2024-04-26T20:27:15.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d0a00640 1 -- 172.21.15.43:0/3758406661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40319d0 con 0x7f96d415f170 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 -- 172.21.15.43:0/3758406661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d415f170 msgr2=0x7f96d40d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 --2- 172.21.15.43:0/3758406661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d415f170 0x7f96d40d2a80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d408d2e0 tx=0x7f96c402a710 comp rx=0 tx=0).stop 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 -- 172.21.15.43:0/3758406661 shutdown_connections 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 --2- 172.21.15.43:0/3758406661 >> v2:172.21.15.164:3300/0 conn(0x7f96d40d5850 0x7f96d40d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 --2- 172.21.15.43:0/3758406661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d415f170 0x7f96d40d2a80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.642+0000 7f96d3200640 1 -- 172.21.15.43:0/3758406661 >> 172.21.15.43:0/3758406661 conn(0x7f96d40d4ca0 msgr2=0x7f96d40d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:15.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.643+0000 7f96d3200640 1 -- 172.21.15.43:0/3758406661 shutdown_connections 2024-04-26T20:27:15.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.643+0000 7f96d3200640 1 -- 172.21.15.43:0/3758406661 wait complete. 2024-04-26T20:27:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.645+0000 7f96d3200640 1 Processor -- start 2024-04-26T20:27:15.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.645+0000 7f96d3200640 1 -- start start 2024-04-26T20:27:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 0x7f96d40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96d415f170 0x7f96d41f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40b5d90 con 0x7f96d40d36a0 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f96d415f550 con 0x7f96d40d5850 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f96d40b5f10 con 0x7f96d415f170 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45478/0 (socket says 172.21.15.43:45478) 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.646+0000 7f96d1e00640 1 -- 172.21.15.43:0/4203914326 learned_addr learned my addr 172.21.15.43:0/4203914326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 -- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96d415f170 msgr2=0x7f96d41f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1400640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 0x7f96d40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d2800640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96d415f170 0x7f96d41f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96d415f170 0x7f96d41f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 -- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 msgr2=0x7f96d40cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 0x7f96d40cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 -- 172.21.15.43:0/4203914326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d4161ca0 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1400640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 0x7f96d40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d1e00640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96c40093e0 tx=0x7f96c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4041020 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c4007820 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d4161f80 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.647+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4036040 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.648+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d415f770 con 0x7f96d40d36a0 2024-04-26T20:27:15.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.649+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d4069a60 con 0x7f96d40d36a0 2024-04-26T20:27:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.650+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96c4044020 con 0x7f96d40d36a0 2024-04-26T20:27:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.651+0000 7f96ca000640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:15.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.651+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f96c40acde0 con 0x7f96d40d36a0 2024-04-26T20:27:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.651+0000 7f96d1400640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.652+0000 7f96d1400640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 0x7f96b006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f96d4160c70 tx=0x7f96bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:15.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.655+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96c404b030 con 0x7f96d40d36a0 2024-04-26T20:27:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.873+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96d406caf0 con 0x7f96d40d36a0 2024-04-26T20:27:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.874+0000 7f96ca000640 1 -- 172.21.15.43:0/4203914326 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f96c407add0 con 0x7f96d40d36a0 2024-04-26T20:27:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 msgr2=0x7f96b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 0x7f96b006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f96d4160c70 tx=0x7f96bc008040 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 msgr2=0x7f96d41616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96c40093e0 tx=0x7f96c4004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 shutdown_connections 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f96b006c8f0 0x7f96b006edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f96d415f170 0x7f96d41f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f96d40d5850 0x7f96d40cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 --2- 172.21.15.43:0/4203914326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96d40d36a0 0x7f96d41616c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 >> 172.21.15.43:0/4203914326 conn(0x7f96d40d4ca0 msgr2=0x7f96d406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 shutdown_connections 2024-04-26T20:27:15.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:15.877+0000 7f96d3200640 1 -- 172.21.15.43:0/4203914326 wait complete. 2024-04-26T20:27:15.896 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:15.896 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:15.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:16.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.174+0000 7f493ca00640 1 Processor -- start 2024-04-26T20:27:16.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.174+0000 7f493ca00640 1 -- start start 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f493ca00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49380d5750 0x7f49380d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f493ca00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49380d0480 con 0x7f49380b5b80 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f493ca00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49380d0720 con 0x7f49380d1620 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f493ca00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f49380d5be0 con 0x7f49380d5750 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f4936a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f49380b5b80 0x7f49380d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58156/0 (socket says 172.21.15.43:58156) 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f4936a00640 1 -- 172.21.15.43:0/1249549667 learned_addr learned my addr 172.21.15.43:0/1249549667 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:16.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.175+0000 7f4937e00640 1 --2- 172.21.15.43:0/1249549667 >> v2:172.21.15.43:3300/0 conn(0x7f49380d5750 0x7f49380d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.176+0000 7f4937e00640 1 -- 172.21.15.43:0/1249549667 >> v1:172.21.15.164:6789/0 conn(0x7f49380d1620 legacy=0x7f49380d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:16.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.176+0000 7f4937e00640 1 -- 172.21.15.43:0/1249549667 >> v1:172.21.15.43:6789/0 conn(0x7f49380b5b80 legacy=0x7f49380d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.176+0000 7f4937e00640 1 -- 172.21.15.43:0/1249549667 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49380d0a00 con 0x7f49380d5750 2024-04-26T20:27:16.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.176+0000 7f4936000640 1 -- 172.21.15.43:0/1249549667 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614228538 0 0) 0x7f49380d0480 con 0x7f49380b5b80 2024-04-26T20:27:16.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.177+0000 7f4937e00640 1 --2- 172.21.15.43:0/1249549667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380d5750 0x7f49380d2f10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4930009d90 tx=0x7f493002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d74a85d3607de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.177+0000 7f4936000640 1 -- 172.21.15.43:0/1249549667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930031600 con 0x7f49380d5750 2024-04-26T20:27:16.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f4936000640 1 -- 172.21.15.43:0/1249549667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49300317a0 con 0x7f49380d5750 2024-04-26T20:27:16.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f4936000640 1 -- 172.21.15.43:0/1249549667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930031b70 con 0x7f49380d5750 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 -- 172.21.15.43:0/1249549667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380d5750 msgr2=0x7f49380d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 --2- 172.21.15.43:0/1249549667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380d5750 0x7f49380d2f10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4930009d90 tx=0x7f493002a610 comp rx=0 tx=0).stop 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 -- 172.21.15.43:0/1249549667 shutdown_connections 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 --2- 172.21.15.43:0/1249549667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380d5750 0x7f49380d2f10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 -- 172.21.15.43:0/1249549667 >> 172.21.15.43:0/1249549667 conn(0x7f49380d4c30 msgr2=0x7f49380d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 -- 172.21.15.43:0/1249549667 shutdown_connections 2024-04-26T20:27:16.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.178+0000 7f493ca00640 1 -- 172.21.15.43:0/1249549667 wait complete. 2024-04-26T20:27:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.180+0000 7f493ca00640 1 Processor -- start 2024-04-26T20:27:16.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.180+0000 7f493ca00640 1 -- start start 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49380d1620 0x7f49381f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49380d5750 0x7f4938152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49380d3450 con 0x7f49380b5b80 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49380d0ce0 con 0x7f49380d5750 2024-04-26T20:27:16.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f493ca00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f49380d35d0 con 0x7f49380d1620 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f4937400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f4937400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41192/0 (socket says 172.21.15.43:41192) 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.181+0000 7f4937400640 1 -- 172.21.15.43:0/4196622999 learned_addr learned my addr 172.21.15.43:0/4196622999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 -- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49380d1620 msgr2=0x7f49381f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49380d1620 0x7f49381f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 -- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49380d5750 msgr2=0x7f4938152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49380d5750 0x7f4938152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 -- 172.21.15.43:0/4196622999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381f4c60 con 0x7f49380b5b80 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f4937400640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f492800c430 tx=0x7f492800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928011790 con 0x7f49380b5b80 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4928017070 con 0x7f49380b5b80 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928011bb0 con 0x7f49380b5b80 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.182+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49380c91b0 con 0x7f49380b5b80 2024-04-26T20:27:16.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.183+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49380c9770 con 0x7f49380b5b80 2024-04-26T20:27:16.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.184+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4938069350 con 0x7f49380b5b80 2024-04-26T20:27:16.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.185+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f492801c460 con 0x7f49380b5b80 2024-04-26T20:27:16.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.186+0000 7f492f400640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.186+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f492808be10 con 0x7f49380b5b80 2024-04-26T20:27:16.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.186+0000 7f4936a00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.187+0000 7f4936a00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4924007fd0 tx=0x7f4924007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.190+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49381f4c60 con 0x7f49380b5b80 2024-04-26T20:27:16.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.399+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49380b3410 con 0x7f49380b5b80 2024-04-26T20:27:16.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.402+0000 7f492f400640 1 -- 172.21.15.43:0/4196622999 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f4928059ea0 con 0x7f49380b5b80 2024-04-26T20:27:16.403 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:16.403 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:16.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 msgr2=0x7f491806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4924007fd0 tx=0x7f4924007920 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 msgr2=0x7f49380b88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f492800c430 tx=0x7f492800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 shutdown_connections 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49380d1620 0x7f49381f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49380d5750 0x7f4938152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 --2- 172.21.15.43:0/4196622999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49380b5b80 0x7f49380b88a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 >> 172.21.15.43:0/4196622999 conn(0x7f49380d4c30 msgr2=0x7f49380d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 shutdown_connections 2024-04-26T20:27:16.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.405+0000 7f493ca00640 1 -- 172.21.15.43:0/4196622999 wait complete. 2024-04-26T20:27:16.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:16.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:16.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.638+0000 7f3f1f200640 1 Processor -- start 2024-04-26T20:27:16.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.638+0000 7f3f1f200640 1 -- start start 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1f200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f180b5410 0x7f3f180b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1f200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f1815d2b0 0x7f3f1815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1f200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180b3900 con 0x7f3f1815a7e0 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1f200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1815abc0 con 0x7f3f180b5410 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1f200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3f180b3b70 con 0x7f3f1815d2b0 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f180b5410 0x7f3f180b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.640+0000 7f3f1de00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3f1815a7e0 0x7f3f180b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58168/0 (socket says 172.21.15.43:58168) 2024-04-26T20:27:16.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1de00640 1 -- 172.21.15.43:0/1192050152 learned_addr learned my addr 172.21.15.43:0/1192050152 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1e800640 1 --2- 172.21.15.43:0/1192050152 >> v2:172.21.15.164:3300/0 conn(0x7f3f1815d2b0 0x7f3f1815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1d400640 1 -- 172.21.15.43:0/1192050152 >> v2:172.21.15.164:3300/0 conn(0x7f3f1815d2b0 msgr2=0x7f3f1815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1d400640 1 --2- 172.21.15.43:0/1192050152 >> v2:172.21.15.164:3300/0 conn(0x7f3f1815d2b0 0x7f3f1815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1d400640 1 -- 172.21.15.43:0/1192050152 >> v1:172.21.15.43:6789/0 conn(0x7f3f1815a7e0 legacy=0x7f3f180b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1d400640 1 -- 172.21.15.43:0/1192050152 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180b3e50 con 0x7f3f180b5410 2024-04-26T20:27:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.641+0000 7f3f1ca00640 1 -- 172.21.15.43:0/1192050152 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479563100 0 0) 0x7f3f180b3900 con 0x7f3f1815a7e0 2024-04-26T20:27:16.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.642+0000 7f3f1d400640 1 --2- 172.21.15.43:0/1192050152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f180b5410 0x7f3f180b5890 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f180cd1c0 tx=0x7f3f0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7017901e7590ff8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1ca00640 1 -- 172.21.15.43:0/1192050152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08037070 con 0x7f3f180b5410 2024-04-26T20:27:16.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1ca00640 1 -- 172.21.15.43:0/1192050152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08032440 con 0x7f3f180b5410 2024-04-26T20:27:16.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1ca00640 1 -- 172.21.15.43:0/1192050152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08032810 con 0x7f3f180b5410 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 -- 172.21.15.43:0/1192050152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f180b5410 msgr2=0x7f3f180b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1192050152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f180b5410 0x7f3f180b5890 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f180cd1c0 tx=0x7f3f0802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 -- 172.21.15.43:0/1192050152 shutdown_connections 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1192050152 >> v2:172.21.15.164:3300/0 conn(0x7f3f1815d2b0 0x7f3f1815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1192050152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f180b5410 0x7f3f180b5890 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 -- 172.21.15.43:0/1192050152 >> 172.21.15.43:0/1192050152 conn(0x7f3f18068090 msgr2=0x7f3f180684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 -- 172.21.15.43:0/1192050152 shutdown_connections 2024-04-26T20:27:16.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.643+0000 7f3f1f200640 1 -- 172.21.15.43:0/1192050152 wait complete. 2024-04-26T20:27:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.645+0000 7f3f1f200640 1 Processor -- start 2024-04-26T20:27:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.645+0000 7f3f1f200640 1 -- start start 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 0x7f3f180b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f1815a7e0 0x7f3f181e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815dc80 con 0x7f3f1815d2b0 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815e010 con 0x7f3f180b5410 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1815e2e0 con 0x7f3f1815a7e0 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.646+0000 7f3f1e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41204/0 (socket says 172.21.15.43:41204) 2024-04-26T20:27:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 -- 172.21.15.43:0/1853149086 learned_addr learned my addr 172.21.15.43:0/1853149086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:16.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1de00640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 0x7f3f180b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1d400640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f1815a7e0 0x7f3f181e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 -- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f1815a7e0 msgr2=0x7f3f181e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f1815a7e0 0x7f3f181e56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 -- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 msgr2=0x7f3f180b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 0x7f3f180b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 -- 172.21.15.43:0/1853149086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180b21f0 con 0x7f3f1815d2b0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1de00640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 0x7f3f180b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.647+0000 7f3f1e800640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400cc60 tx=0x7f3f1400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.648+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14003990 con 0x7f3f1815d2b0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.648+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f14003b30 con 0x7f3f1815d2b0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.648+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14004020 con 0x7f3f1815d2b0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.648+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f180b24d0 con 0x7f3f1815d2b0 2024-04-26T20:27:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.648+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f180b2a90 con 0x7f3f1815d2b0 2024-04-26T20:27:16.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.649+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f180506c0 con 0x7f3f1815d2b0 2024-04-26T20:27:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.650+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f140041c0 con 0x7f3f1815d2b0 2024-04-26T20:27:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.651+0000 7f3f12000640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:16.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.651+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f3f1408c3a0 con 0x7f3f1815d2b0 2024-04-26T20:27:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.651+0000 7f3f1de00640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.652+0000 7f3f1de00640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 0x7f3efc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f60 tx=0x7f3f0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.656+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f14014030 con 0x7f3f1815d2b0 2024-04-26T20:27:16.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.858+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f180af830 con 0x7f3f1815d2b0 2024-04-26T20:27:16.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.859+0000 7f3f12000640 1 -- 172.21.15.43:0/1853149086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3f1405a430 con 0x7f3f1815d2b0 2024-04-26T20:27:16.860 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 msgr2=0x7f3efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 0x7f3efc06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c000f60 tx=0x7f3f0c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 msgr2=0x7f3f180d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400cc60 tx=0x7f3f1400a5c0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 shutdown_connections 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3efc06c8f0 0x7f3efc06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f1815a7e0 0x7f3f181e56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f1815d2b0 0x7f3f180d0040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 --2- 172.21.15.43:0/1853149086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f180b5410 0x7f3f180b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 >> 172.21.15.43:0/1853149086 conn(0x7f3f18068090 msgr2=0x7f3f180cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 shutdown_connections 2024-04-26T20:27:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:16.862+0000 7f3f1f200640 1 -- 172.21.15.43:0/1853149086 wait complete. 2024-04-26T20:27:16.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:17.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.091+0000 7f7767200640 1 Processor -- start 2024-04-26T20:27:17.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.091+0000 7f7767200640 1 -- start start 2024-04-26T20:27:17.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f776015b980 0x7f776006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f776015bd60 0x7f776006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 0x7f77600b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f776015d060 con 0x7f776015bd60 2024-04-26T20:27:17.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f77600b4fe0 con 0x7f776015b980 2024-04-26T20:27:17.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7767200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f776015c2a0 con 0x7f77600b4610 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7765400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f776015bd60 0x7f776006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7765400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f776015bd60 0x7f776006f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41218/0 (socket says 172.21.15.43:41218) 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.093+0000 7f7765400640 1 -- 172.21.15.43:0/2476728974 learned_addr learned my addr 172.21.15.43:0/2476728974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7766800640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 0x7f77600b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765e00640 1 --2- 172.21.15.43:0/2476728974 >> v2:172.21.15.164:3300/0 conn(0x7f776015b980 0x7f776006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765400640 1 -- 172.21.15.43:0/2476728974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 msgr2=0x7f77600b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765400640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 0x7f77600b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765400640 1 -- 172.21.15.43:0/2476728974 >> v2:172.21.15.164:3300/0 conn(0x7f776015b980 msgr2=0x7f776006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765400640 1 --2- 172.21.15.43:0/2476728974 >> v2:172.21.15.164:3300/0 conn(0x7f776015b980 0x7f776006ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7765400640 1 -- 172.21.15.43:0/2476728974 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776015c580 con 0x7f776015bd60 2024-04-26T20:27:17.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.094+0000 7f7766800640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 0x7f77600b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:17.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.095+0000 7f7765400640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776015bd60 0x7f776006f1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f776006fe40 tx=0x7f775002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0c95646a1911d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.095+0000 7f7764a00640 1 -- 172.21.15.43:0/2476728974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750032790 con 0x7f776015bd60 2024-04-26T20:27:17.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7764a00640 1 -- 172.21.15.43:0/2476728974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750038070 con 0x7f776015bd60 2024-04-26T20:27:17.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7764a00640 1 -- 172.21.15.43:0/2476728974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750032b50 con 0x7f776015bd60 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 -- 172.21.15.43:0/2476728974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776015bd60 msgr2=0x7f776006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776015bd60 0x7f776006f1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f776006fe40 tx=0x7f775002c890 comp rx=0 tx=0).stop 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 -- 172.21.15.43:0/2476728974 shutdown_connections 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77600b4610 0x7f77600b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 --2- 172.21.15.43:0/2476728974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f776015bd60 0x7f776006f1d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 --2- 172.21.15.43:0/2476728974 >> v2:172.21.15.164:3300/0 conn(0x7f776015b980 0x7f776006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 -- 172.21.15.43:0/2476728974 >> 172.21.15.43:0/2476728974 conn(0x7f7760068090 msgr2=0x7f77600684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 -- 172.21.15.43:0/2476728974 shutdown_connections 2024-04-26T20:27:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.096+0000 7f7767200640 1 -- 172.21.15.43:0/2476728974 wait complete. 2024-04-26T20:27:17.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.098+0000 7f7767200640 1 Processor -- start 2024-04-26T20:27:17.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.098+0000 7f7767200640 1 -- start start 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 0x7f77600aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 0x7f77600ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77600b52b0 con 0x7f77600b4610 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f776015cb40 con 0x7f776015bd60 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7767200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7760072cd0 con 0x7f776015b980 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.099+0000 7f7765e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41224/0 (socket says 172.21.15.43:41224) 2024-04-26T20:27:17.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 -- 172.21.15.43:0/2409966512 learned_addr learned my addr 172.21.15.43:0/2409966512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:17.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765400640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 0x7f77600aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7766800640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 0x7f77600ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 -- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 msgr2=0x7f77600aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 0x7f77600aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 -- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 msgr2=0x7f77600ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 0x7f77600ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 -- 172.21.15.43:0/2409966512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77600adad0 con 0x7f77600b4610 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765400640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 0x7f77600aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7766800640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 0x7f77600ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.100+0000 7f7765e00640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f775400c7f0 tx=0x7f775400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.101+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754015070 con 0x7f77600b4610 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.101+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754004240 con 0x7f77600b4610 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.101+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754007620 con 0x7f77600b4610 2024-04-26T20:27:17.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.101+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77600addb0 con 0x7f77600b4610 2024-04-26T20:27:17.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.101+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77600ae370 con 0x7f77600b4610 2024-04-26T20:27:17.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.102+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77600506c0 con 0x7f77600b4610 2024-04-26T20:27:17.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.103+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77540079d0 con 0x7f77600b4610 2024-04-26T20:27:17.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.104+0000 7f775a000640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.104+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f775408bff0 con 0x7f77600b4610 2024-04-26T20:27:17.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.104+0000 7f7765400640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.105+0000 7f7765400640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 0x7f774406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7750037000 tx=0x7f7750032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.108+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f775405a000 con 0x7f77600b4610 2024-04-26T20:27:17.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.312+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f77601578d0 con 0x7f77600b4610 2024-04-26T20:27:17.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.312+0000 7f775a000640 1 -- 172.21.15.43:0/2409966512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7754059900 con 0x7f77600b4610 2024-04-26T20:27:17.313 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 msgr2=0x7f774406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 0x7f774406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7750037000 tx=0x7f7750032000 comp rx=0 tx=0).stop 2024-04-26T20:27:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 msgr2=0x7f77600b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f775400c7f0 tx=0x7f775400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 shutdown_connections 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f774406c8f0 0x7f774406edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f776015b980 0x7f77600aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f776015bd60 0x7f77600ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 --2- 172.21.15.43:0/2409966512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77600b4610 0x7f77600b2670 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 >> 172.21.15.43:0/2409966512 conn(0x7f7760068090 msgr2=0x7f776006db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 shutdown_connections 2024-04-26T20:27:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.315+0000 7f7767200640 1 -- 172.21.15.43:0/2409966512 wait complete. 2024-04-26T20:27:17.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.552+0000 7fdc1ae00640 1 Processor -- start 2024-04-26T20:27:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.553+0000 7fdc1ae00640 1 -- start start 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.554+0000 7fdc1ae00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdc1415cae0 0x7fdc140b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.554+0000 7fdc1ae00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1415f480 con 0x7fdc140d12e0 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.554+0000 7fdc1ae00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1415f760 con 0x7fdc140b5be0 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.554+0000 7fdc1ae00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdc14160090 con 0x7fdc1415cae0 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.555+0000 7fdc19000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc140d12e0 0x7fdc1415c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58184/0 (socket says 172.21.15.43:58184) 2024-04-26T20:27:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.555+0000 7fdc19000640 1 -- 172.21.15.43:0/4182547740 learned_addr learned my addr 172.21.15.43:0/4182547740 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.555+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639800507 0 0) 0x7fdc1415f480 con 0x7fdc140d12e0 2024-04-26T20:27:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.555+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbfc003520 con 0x7fdc140d12e0 2024-04-26T20:27:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.555+0000 7fdc1a400640 1 --2- 172.21.15.43:0/4182547740 >> v2:172.21.15.164:3300/0 conn(0x7fdc1415cae0 0x7fdc140b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 734861225 0 0) 0x7fdbfc003520 con 0x7fdc140d12e0 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 >> v2:172.21.15.164:3300/0 conn(0x7fdc1415cae0 msgr2=0x7fdc140b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 --2- 172.21.15.43:0/4182547740 >> v2:172.21.15.164:3300/0 conn(0x7fdc1415cae0 0x7fdc140b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 >> v1:172.21.15.164:6789/0 conn(0x7fdc140b5be0 legacy=0x7fdc140d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1415fa40 con 0x7fdc140d12e0 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fdc00001670 con 0x7fdc140d12e0 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 >> v1:172.21.15.43:6789/0 conn(0x7fdc140d12e0 legacy=0x7fdc1415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc003a70 0x7fdbfc003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbfc0052e0 0x7fdbfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14160090 con 0x7fdbfc004800 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc00c1e0 con 0x7fdbfc003a70 2024-04-26T20:27:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc00c3e0 con 0x7fdbfc0052e0 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.556+0000 7fdc19000640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19a00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc003a70 0x7fdbfc003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc1a400640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbfc0052e0 0x7fdbfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19000640 1 -- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbfc0052e0 msgr2=0x7fdbfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19000640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdbfc0052e0 0x7fdbfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19000640 1 -- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc003a70 msgr2=0x7fdbfc003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19000640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc003a70 0x7fdbfc003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc19000640 1 -- 172.21.15.43:0/4182547740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc003520 con 0x7fdbfc004800 2024-04-26T20:27:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.557+0000 7fdc1a400640 1 -- 172.21.15.43:0/4182547740 reap_dead start 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.558+0000 7fdc19000640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdbfc004c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c0e0 tx=0x7fdbfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954edf0b84f1c085 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.558+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00007c20 con 0x7fdbfc004800 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.558+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00005af0 con 0x7fdbfc004800 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.558+0000 7fdc13e00640 1 -- 172.21.15.43:0/4182547740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00007070 con 0x7fdbfc004800 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 -- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 msgr2=0x7fdbfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdbfc004c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000c0e0 tx=0x7fdbfc005830 comp rx=0 tx=0).stop 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 -- 172.21.15.43:0/4182547740 shutdown_connections 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/4182547740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 -- 172.21.15.43:0/4182547740 >> 172.21.15.43:0/4182547740 conn(0x7fdc140684e0 msgr2=0x7fdc140688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 -- 172.21.15.43:0/4182547740 shutdown_connections 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.559+0000 7fdc1ae00640 1 -- 172.21.15.43:0/4182547740 wait complete. 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.561+0000 7fdc1ae00640 1 Processor -- start 2024-04-26T20:27:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.561+0000 7fdc1ae00640 1 -- start start 2024-04-26T20:27:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc14154880 0x7fdc1414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc141fc6e0 0x7fdc140b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140b6140 con 0x7fdbfc004800 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140b5fc0 con 0x7fdc141fc6e0 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc1ae00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc140b62c0 con 0x7fdc14154880 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc19a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc19a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41248/0 (socket says 172.21.15.43:41248) 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.562+0000 7fdc19a00640 1 -- 172.21.15.43:0/2984080591 learned_addr learned my addr 172.21.15.43:0/2984080591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19000640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc14154880 0x7fdc1414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc1a400640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc141fc6e0 0x7fdc140b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 -- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc14154880 msgr2=0x7fdc1414d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc14154880 0x7fdc1414d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 -- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc141fc6e0 msgr2=0x7fdc140b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc141fc6e0 0x7fdc140b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 -- 172.21.15.43:0/2984080591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc140cea60 con 0x7fdbfc004800 2024-04-26T20:27:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc19a00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdc08007810 tx=0x7fdc0800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.563+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080050d0 con 0x7fdbfc004800 2024-04-26T20:27:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.564+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc08018070 con 0x7fdbfc004800 2024-04-26T20:27:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.564+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc141f9990 con 0x7fdbfc004800 2024-04-26T20:27:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.564+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0800d6d0 con 0x7fdbfc004800 2024-04-26T20:27:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.564+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc140cecd0 con 0x7fdbfc004800 2024-04-26T20:27:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.565+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc140506c0 con 0x7fdbfc004800 2024-04-26T20:27:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.566+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc0800d870 con 0x7fdbfc004800 2024-04-26T20:27:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.567+0000 7fdc11600640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 0x7fdbf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.567+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fdc0808bcc0 con 0x7fdbfc004800 2024-04-26T20:27:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.567+0000 7fdc19000640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 0x7fdbf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.568+0000 7fdc19000640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 0x7fdbf406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc00032000 tx=0x7fdc00009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.572+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc08092080 con 0x7fdbfc004800 2024-04-26T20:27:17.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.778+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc140ccdd0 con 0x7fdbfc004800 2024-04-26T20:27:17.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.779+0000 7fdc11600640 1 -- 172.21.15.43:0/2984080591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fdc08059c20 con 0x7fdbfc004800 2024-04-26T20:27:17.780 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 msgr2=0x7fdbf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 0x7fdbf406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc00032000 tx=0x7fdc00009030 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 msgr2=0x7fdc141f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdc08007810 tx=0x7fdc0800fc00 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 shutdown_connections 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdbf406c920 0x7fdbf406ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc14154880 0x7fdc1414d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc141fc6e0 0x7fdc140b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 --2- 172.21.15.43:0/2984080591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbfc004800 0x7fdc141f90f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.781+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 >> 172.21.15.43:0/2984080591 conn(0x7fdc140684e0 msgr2=0x7fdc1406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.782+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 shutdown_connections 2024-04-26T20:27:17.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:17.782+0000 7fdc1ae00640 1 -- 172.21.15.43:0/2984080591 wait complete. 2024-04-26T20:27:17.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.021+0000 7f0dbb200640 1 Processor -- start 2024-04-26T20:27:18.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.021+0000 7f0dbb200640 1 -- start start 2024-04-26T20:27:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dbb200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0dbc15ef30 0x7f0dbc0d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dbb200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dbc0d3300 0x7f0dbc162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dbb200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dbc0d0d10 con 0x7f0dbc15f310 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dbb200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0dbc0d3a70 con 0x7f0dbc0d3300 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dbb200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0dbc0cfb20 con 0x7f0dbc15ef30 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dba800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dbc0d3300 0x7f0dbc162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dba800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dbc0d3300 0x7f0dbc162c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41260/0 (socket says 172.21.15.43:41260) 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.023+0000 7f0dba800640 1 -- 172.21.15.43:0/2135364563 learned_addr learned my addr 172.21.15.43:0/2135364563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:18.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.024+0000 7f0db9e00640 1 --2- 172.21.15.43:0/2135364563 >> v2:172.21.15.164:3300/0 conn(0x7f0dbc15ef30 0x7f0dbc0d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.024+0000 7f0dba800640 1 -- 172.21.15.43:0/2135364563 >> v2:172.21.15.164:3300/0 conn(0x7f0dbc15ef30 msgr2=0x7f0dbc0d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.024+0000 7f0dba800640 1 --2- 172.21.15.43:0/2135364563 >> v2:172.21.15.164:3300/0 conn(0x7f0dbc15ef30 0x7f0dbc0d5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.024+0000 7f0dba800640 1 -- 172.21.15.43:0/2135364563 >> v1:172.21.15.164:6789/0 conn(0x7f0dbc15f310 legacy=0x7f0dbc0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.024+0000 7f0dba800640 1 -- 172.21.15.43:0/2135364563 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc0cfe00 con 0x7f0dbc0d3300 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.025+0000 7f0dba800640 1 --2- 172.21.15.43:0/2135364563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc162c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc06c5a0 tx=0x7f0db402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88e4ff2ac35f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.025+0000 7f0db8a00640 1 -- 172.21.15.43:0/2135364563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4036070 con 0x7f0dbc0d3300 2024-04-26T20:27:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.025+0000 7f0db8a00640 1 -- 172.21.15.43:0/2135364563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db4031600 con 0x7f0dbc0d3300 2024-04-26T20:27:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.025+0000 7f0db8a00640 1 -- 172.21.15.43:0/2135364563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db40319d0 con 0x7f0dbc0d3300 2024-04-26T20:27:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 -- 172.21.15.43:0/2135364563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc0d3300 msgr2=0x7f0dbc162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 --2- 172.21.15.43:0/2135364563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc162c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc06c5a0 tx=0x7f0db402a660 comp rx=0 tx=0).stop 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 -- 172.21.15.43:0/2135364563 shutdown_connections 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 --2- 172.21.15.43:0/2135364563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc162c00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 --2- 172.21.15.43:0/2135364563 >> v2:172.21.15.164:3300/0 conn(0x7f0dbc15ef30 0x7f0dbc0d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 -- 172.21.15.43:0/2135364563 >> 172.21.15.43:0/2135364563 conn(0x7f0dbc0d4d20 msgr2=0x7f0dbc0d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 -- 172.21.15.43:0/2135364563 shutdown_connections 2024-04-26T20:27:18.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.026+0000 7f0dbb200640 1 -- 172.21.15.43:0/2135364563 wait complete. 2024-04-26T20:27:18.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.028+0000 7f0dbb200640 1 Processor -- start 2024-04-26T20:27:18.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.028+0000 7f0dbb200640 1 -- start start 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 0x7f0dbc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc0d3710 con 0x7f0dbc15ef30 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc0cf7f0 con 0x7f0dbc0d3300 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.029+0000 7f0dbb200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0dbc163140 con 0x7f0dbc1593a0 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41272/0 (socket says 172.21.15.43:41272) 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 -- 172.21.15.43:0/634348408 learned_addr learned my addr 172.21.15.43:0/634348408 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9e00640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0dba800640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 0x7f0dbc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 -- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 msgr2=0x7f0dbc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 0x7f0dbc0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 -- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 msgr2=0x7f0dbc14f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc14f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9400640 1 -- 172.21.15.43:0/634348408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc14f830 con 0x7f0dbc15ef30 2024-04-26T20:27:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0dba800640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 0x7f0dbc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.030+0000 7f0db9e00640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0db9400640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b3f0 tx=0x7f0da800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8015020 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8011830 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8011c20 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc161400 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.031+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1619c0 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.032+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc0506c0 con 0x7f0dbc15ef30 2024-04-26T20:27:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.033+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0da8018020 con 0x7f0dbc15ef30 2024-04-26T20:27:18.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.034+0000 7f0db2000640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 0x7f0d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.034+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f0da808bbf0 con 0x7f0dbc15ef30 2024-04-26T20:27:18.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.035+0000 7f0db9e00640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 0x7f0d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.035+0000 7f0db9e00640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 0x7f0d9c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00a9d0 tx=0x7f0dac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.039+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0da801f030 con 0x7f0dbc15ef30 2024-04-26T20:27:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.244+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0dbc0b6320 con 0x7f0dbc15ef30 2024-04-26T20:27:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.245+0000 7f0db2000640 1 -- 172.21.15.43:0/634348408 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f0dbc14f830 con 0x7f0dbc15ef30 2024-04-26T20:27:18.245 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 msgr2=0x7f0d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 0x7f0d9c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00a9d0 tx=0x7f0dac008040 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 msgr2=0x7f0dbc160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b3f0 tx=0x7f0da800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 shutdown_connections 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0d9c06c8f0 0x7f0d9c06edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dbc1593a0 0x7f0dbc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15ef30 0x7f0dbc160bb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 --2- 172.21.15.43:0/634348408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dbc0d3300 0x7f0dbc14f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 >> 172.21.15.43:0/634348408 conn(0x7f0dbc0d4d20 msgr2=0x7f0dbc15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 shutdown_connections 2024-04-26T20:27:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.248+0000 7f0dbb200640 1 -- 172.21.15.43:0/634348408 wait complete. 2024-04-26T20:27:18.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.478+0000 7f166b200640 1 Processor -- start 2024-04-26T20:27:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.479+0000 7f166b200640 1 -- start start 2024-04-26T20:27:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.480+0000 7f166b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f16640d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.480+0000 7f166b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f16640d05b0 0x7f16640d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.480+0000 7f166b200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16640d0de0 con 0x7f16640d3db0 2024-04-26T20:27:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.480+0000 7f166b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f16640d2ab0 con 0x7f16640d05b0 2024-04-26T20:27:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.480+0000 7f166b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f16640d2d90 con 0x7f166415f180 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1669200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16640d3db0 0x7f1664162d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58188/0 (socket says 172.21.15.43:58188) 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1669200640 1 -- 172.21.15.43:0/2057415015 learned_addr learned my addr 172.21.15.43:0/2057415015 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1663e00640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f16640d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1663400640 1 --2- 172.21.15.43:0/2057415015 >> v2:172.21.15.164:3300/0 conn(0x7f16640d05b0 0x7f16640d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196035297 0 0) 0x7f16640d0de0 con 0x7f16640d3db0 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.481+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1648003520 con 0x7f16640d3db0 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1975380882 0 0) 0x7f1648003520 con 0x7f16640d3db0 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 msgr2=0x7f16640d5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f16640d5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 >> v2:172.21.15.164:3300/0 conn(0x7f16640d05b0 msgr2=0x7f16640d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 --2- 172.21.15.43:0/2057415015 >> v2:172.21.15.164:3300/0 conn(0x7f16640d05b0 0x7f16640d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640d3070 con 0x7f16640d3db0 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f165c001670 con 0x7f16640d3db0 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 >> v1:172.21.15.43:6789/0 conn(0x7f16640d3db0 legacy=0x7f1664162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16480039b0 0x7f1648003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f1648004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f16480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16640d2ab0 con 0x7f1648004d40 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f16640d2d90 con 0x7f16480039b0 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f164800ba80 con 0x7f166415f180 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1663400640 1 --2- 172.21.15.43:0/2057415015 >> v2:172.21.15.164:3300/0 conn(0x7f16640d05b0 0x7f16640d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.482+0000 7f1669200640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f16480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1663400640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f1648004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1669200640 1 -- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 msgr2=0x7f1648004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1669200640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f166415f180 0x7f1648004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1669200640 1 -- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16480039b0 msgr2=0x7f1648003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:18.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1669200640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16480039b0 0x7f1648003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.483+0000 7f1669200640 1 -- 172.21.15.43:0/2057415015 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1648003520 con 0x7f1648004d40 2024-04-26T20:27:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f1669200640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f16480051c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c240 tx=0x7f1648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e38297e9979735e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c02e070 con 0x7f1648004d40 2024-04-26T20:27:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c005af0 con 0x7f1648004d40 2024-04-26T20:27:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f1662a00640 1 -- 172.21.15.43:0/2057415015 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c007d30 con 0x7f1648004d40 2024-04-26T20:27:18.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f166b200640 1 -- 172.21.15.43:0/2057415015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 msgr2=0x7f16480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f166b200640 1 --2- 172.21.15.43:0/2057415015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f16480051c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c240 tx=0x7f1648005830 comp rx=0 tx=0).stop 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.484+0000 7f1669200640 1 -- 172.21.15.43:0/2057415015 reap_dead start 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.485+0000 7f166b200640 1 -- 172.21.15.43:0/2057415015 shutdown_connections 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.485+0000 7f166b200640 1 -- 172.21.15.43:0/2057415015 >> 172.21.15.43:0/2057415015 conn(0x7f16640d4ee0 msgr2=0x7f16640d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.485+0000 7f166b200640 1 -- 172.21.15.43:0/2057415015 shutdown_connections 2024-04-26T20:27:18.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.485+0000 7f166b200640 1 -- 172.21.15.43:0/2057415015 wait complete. 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.486+0000 7f166b200640 1 Processor -- start 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.487+0000 7f166b200640 1 -- start start 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.487+0000 7f166b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f166b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 0x7f16641e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f166b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16641e8f60 0x7f16640c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f166b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16640d09d0 con 0x7f1648004d40 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f166b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f166415f560 con 0x7f16640cbfb0 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f166b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f16640d0b50 con 0x7f16641e8f60 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41292/0 (socket says 172.21.15.43:41292) 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 -- 172.21.15.43:0/2118096581 learned_addr learned my addr 172.21.15.43:0/2118096581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1669200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16641e8f60 0x7f16640c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 -- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16641e8f60 msgr2=0x7f16640c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663400640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 0x7f16641e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16641e8f60 0x7f16640c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 -- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 msgr2=0x7f16641e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 0x7f16641e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663e00640 1 -- 172.21.15.43:0/2118096581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16641ea7b0 con 0x7f1648004d40 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.488+0000 7f1663400640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 0x7f16641e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:18.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f1663e00640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f164c00e7f0 tx=0x7f164c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c017070 con 0x7f1648004d40 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c00c8c0 con 0x7f1648004d40 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16641eaa90 con 0x7f1648004d40 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c00ccb0 con 0x7f1648004d40 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.489+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16641e9a80 con 0x7f1648004d40 2024-04-26T20:27:18.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.490+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1664076460 con 0x7f1648004d40 2024-04-26T20:27:18.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.492+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f164c01a070 con 0x7f1648004d40 2024-04-26T20:27:18.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.493+0000 7f165be00640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:18.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.493+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f164c08c0f0 con 0x7f1648004d40 2024-04-26T20:27:18.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.493+0000 7f1663400640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.494+0000 7f1663400640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1654009540 tx=0x7f165400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:18.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.497+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1664076460 con 0x7f1648004d40 2024-04-26T20:27:18.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.715+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1664069a60 con 0x7f1648004d40 2024-04-26T20:27:18.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.716+0000 7f165be00640 1 -- 172.21.15.43:0/2118096581 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f164c05a180 con 0x7f1648004d40 2024-04-26T20:27:18.716 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.718+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 msgr2=0x7f164006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.718+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 0x7f164006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1654009540 tx=0x7f165400b040 comp rx=0 tx=0).stop 2024-04-26T20:27:18.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 msgr2=0x7f1664154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f164c00e7f0 tx=0x7f164c00ecb0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 shutdown_connections 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f164006c8f0 0x7f164006edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16641e8f60 0x7f16640c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16640cbfb0 0x7f16641e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 --2- 172.21.15.43:0/2118096581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1648004d40 0x7f1664154140 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 >> 172.21.15.43:0/2118096581 conn(0x7f16640d4ee0 msgr2=0x7f16640b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 shutdown_connections 2024-04-26T20:27:18.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:18.719+0000 7f166b200640 1 -- 172.21.15.43:0/2118096581 wait complete. 2024-04-26T20:27:18.739 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2024-04-26T20:27:18.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:18.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:27:18.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:18.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:27:18.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:18.835 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:27:18.903 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:18.904 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:18.965 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:18.965 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:19.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.228+0000 7fd9d1600640 1 Processor -- start 2024-04-26T20:27:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.229+0000 7fd9d1600640 1 -- start start 2024-04-26T20:27:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9d1600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9d1600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9cc0cfe60 0x7fd9cc16d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9d1600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9cc14d290 con 0x7fd9cc0d1570 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9d1600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd9cc0b5140 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9d1600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd9cc0b5420 con 0x7fd9cc0cfe60 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.230+0000 7fd9caa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41302/0 (socket says 172.21.15.43:41302) 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 -- 172.21.15.43:0/809131793 learned_addr learned my addr 172.21.15.43:0/809131793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9cb400640 1 --2- 172.21.15.43:0/809131793 >> v2:172.21.15.164:3300/0 conn(0x7fd9cc0cfe60 0x7fd9cc16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 -- 172.21.15.43:0/809131793 >> v2:172.21.15.164:3300/0 conn(0x7fd9cc0cfe60 msgr2=0x7fd9cc16d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 --2- 172.21.15.43:0/809131793 >> v2:172.21.15.164:3300/0 conn(0x7fd9cc0cfe60 0x7fd9cc16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 -- 172.21.15.43:0/809131793 >> v1:172.21.15.164:6789/0 conn(0x7fd9cc0d1570 legacy=0x7fd9cc0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.231+0000 7fd9caa00640 1 -- 172.21.15.43:0/809131793 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc0b5700 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.232+0000 7fd9caa00640 1 --2- 172.21.15.43:0/809131793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0cf4f0 tx=0x7fd9bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb6aad8a43acfe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.232+0000 7fd9ca000640 1 -- 172.21.15.43:0/809131793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc037070 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9ca000640 1 -- 172.21.15.43:0/809131793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9bc032440 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9ca000640 1 -- 172.21.15.43:0/809131793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc032810 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 -- 172.21.15.43:0/809131793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc14ceb0 msgr2=0x7fd9cc0b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 --2- 172.21.15.43:0/809131793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0cf4f0 tx=0x7fd9bc02a230 comp rx=0 tx=0).stop 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 -- 172.21.15.43:0/809131793 shutdown_connections 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 --2- 172.21.15.43:0/809131793 >> v2:172.21.15.164:3300/0 conn(0x7fd9cc0cfe60 0x7fd9cc16d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 --2- 172.21.15.43:0/809131793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0b4a60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 -- 172.21.15.43:0/809131793 >> 172.21.15.43:0/809131793 conn(0x7fd9cc0b2900 msgr2=0x7fd9cc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 -- 172.21.15.43:0/809131793 shutdown_connections 2024-04-26T20:27:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.233+0000 7fd9d1600640 1 -- 172.21.15.43:0/809131793 wait complete. 2024-04-26T20:27:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.235+0000 7fd9d1600640 1 Processor -- start 2024-04-26T20:27:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.235+0000 7fd9d1600640 1 -- start start 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9cc0cfe60 0x7fd9cc162e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0d1a00 con 0x7fd9cc0d1570 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0b5cc0 con 0x7fd9cc14ceb0 2024-04-26T20:27:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9d1600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc06f000 con 0x7fd9cc0cfe60 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41306/0 (socket says 172.21.15.43:41306) 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.236+0000 7fd9c3e00640 1 -- 172.21.15.43:0/746383792 learned_addr learned my addr 172.21.15.43:0/746383792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9cb400640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9caa00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9cc0cfe60 0x7fd9cc162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 -- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9cc0cfe60 msgr2=0x7fd9cc162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9cc0cfe60 0x7fd9cc162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 -- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 msgr2=0x7fd9cc0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 -- 172.21.15.43:0/746383792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc1654c0 con 0x7fd9cc0d1570 2024-04-26T20:27:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9cb400640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c3e00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400c460 tx=0x7fd9b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.237+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4016020 con 0x7fd9cc0d1570 2024-04-26T20:27:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.238+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4012870 con 0x7fd9cc0d1570 2024-04-26T20:27:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.238+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc1657a0 con 0x7fd9cc0d1570 2024-04-26T20:27:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.238+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4012c60 con 0x7fd9cc0d1570 2024-04-26T20:27:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.238+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc165d60 con 0x7fd9cc0d1570 2024-04-26T20:27:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.239+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc069350 con 0x7fd9cc0d1570 2024-04-26T20:27:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.240+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9b4019020 con 0x7fd9cc0d1570 2024-04-26T20:27:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.241+0000 7fd9c2a00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 0x7fd9a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.241+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd9b408bcd0 con 0x7fd9cc0d1570 2024-04-26T20:27:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.241+0000 7fd9caa00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 0x7fd9a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.242+0000 7fd9caa00640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 0x7fd9a806ee00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc034000 tx=0x7fd9bc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.245+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9b4020030 con 0x7fd9cc0d1570 2024-04-26T20:27:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.451+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9cc0b1390 con 0x7fd9cc0d1570 2024-04-26T20:27:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.455+0000 7fd9c2a00640 1 -- 172.21.15.43:0/746383792 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fd9b4059cb0 con 0x7fd9cc0d1570 2024-04-26T20:27:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:19.456 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:19.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 msgr2=0x7fd9a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 0x7fd9a806ee00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc034000 tx=0x7fd9bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 msgr2=0x7fd9cc164ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400c460 tx=0x7fd9b400c920 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 shutdown_connections 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd9a806c940 0x7fd9a806ee00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9cc14ceb0 0x7fd9cc0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0d1570 0x7fd9cc164ae0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 --2- 172.21.15.43:0/746383792 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9cc0cfe60 0x7fd9cc162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 >> 172.21.15.43:0/746383792 conn(0x7fd9cc0b2900 msgr2=0x7fd9cc0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 shutdown_connections 2024-04-26T20:27:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.457+0000 7fd9d1600640 1 -- 172.21.15.43:0/746383792 wait complete. 2024-04-26T20:27:19.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:19.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.693+0000 7f5327e00640 1 Processor -- start 2024-04-26T20:27:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.694+0000 7f5327e00640 1 -- start start 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.695+0000 7f5327e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f532815a7e0 0x7f53280b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.695+0000 7f5327e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.695+0000 7f5327e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53280b3900 con 0x7f53280b5410 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.695+0000 7f5327e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f532815abc0 con 0x7f532815a7e0 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.695+0000 7f5327e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53280b3b70 con 0x7f532815d9c0 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f532815a7e0 0x7f53280b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f532815a7e0 0x7f53280b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41310/0 (socket says 172.21.15.43:41310) 2024-04-26T20:27:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 -- 172.21.15.43:0/3723884356 learned_addr learned my addr 172.21.15.43:0/3723884356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5327400640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 -- 172.21.15.43:0/3723884356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 msgr2=0x7f53280b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 -- 172.21.15.43:0/3723884356 >> v1:172.21.15.164:6789/0 conn(0x7f53280b5410 legacy=0x7f532815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5326a00640 1 -- 172.21.15.43:0/3723884356 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280b3e50 con 0x7f532815a7e0 2024-04-26T20:27:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.696+0000 7f5327400640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.697+0000 7f5326a00640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53280b4650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7540dc9395a0fda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.698+0000 7f5325600640 1 -- 172.21.15.43:0/3723884356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318037070 con 0x7f532815a7e0 2024-04-26T20:27:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.698+0000 7f5325600640 1 -- 172.21.15.43:0/3723884356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318032440 con 0x7f532815a7e0 2024-04-26T20:27:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.698+0000 7f5325600640 1 -- 172.21.15.43:0/3723884356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318032810 con 0x7f532815a7e0 2024-04-26T20:27:19.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.698+0000 7f5327e00640 1 -- 172.21.15.43:0/3723884356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 msgr2=0x7f53280b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.698+0000 7f5327e00640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53280b4650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531802a210 comp rx=0 tx=0).stop 2024-04-26T20:27:19.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 -- 172.21.15.43:0/3723884356 shutdown_connections 2024-04-26T20:27:19.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 --2- 172.21.15.43:0/3723884356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53280b4650 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 -- 172.21.15.43:0/3723884356 >> 172.21.15.43:0/3723884356 conn(0x7f5328068090 msgr2=0x7f53280684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:19.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 -- 172.21.15.43:0/3723884356 shutdown_connections 2024-04-26T20:27:19.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.699+0000 7f5327e00640 1 -- 172.21.15.43:0/3723884356 wait complete. 2024-04-26T20:27:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.700+0000 7f5327e00640 1 Processor -- start 2024-04-26T20:27:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.701+0000 7f5327e00640 1 -- start start 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b5410 0x7f53281e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f53280b4b90 con 0x7f532815a7e0 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f532815e010 con 0x7f53280b5410 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5327e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f532815e2e0 con 0x7f532815d9c0 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41324/0 (socket says 172.21.15.43:41324) 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 -- 172.21.15.43:0/2884471176 learned_addr learned my addr 172.21.15.43:0/2884471176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 -- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 msgr2=0x7f53280ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326a00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b5410 0x7f53281e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 -- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b5410 msgr2=0x7f53281e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b5410 0x7f53281e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.702+0000 7f5326000640 1 -- 172.21.15.43:0/2884471176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53281e8000 con 0x7f532815a7e0 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f5326000640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f531000c430 tx=0x7f531000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310011790 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5310017070 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310011bb0 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5328147f90 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.703+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5328148550 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.704+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53280506c0 con 0x7f532815a7e0 2024-04-26T20:27:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.705+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f531001c460 con 0x7f532815a7e0 2024-04-26T20:27:19.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.706+0000 7f531f400640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 0x7f530406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:19.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.706+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f531008bf50 con 0x7f532815a7e0 2024-04-26T20:27:19.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.707+0000 7f5326a00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 0x7f530406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:19.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.707+0000 7f5326a00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 0x7f530406ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5318034000 tx=0x7f531802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.711+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5310059e20 con 0x7f532815a7e0 2024-04-26T20:27:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.906+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f532815f4b0 con 0x7f532815a7e0 2024-04-26T20:27:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.906+0000 7f531f400640 1 -- 172.21.15.43:0/2884471176 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5310059720 con 0x7f532815a7e0 2024-04-26T20:27:19.907 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 msgr2=0x7f530406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 0x7f530406ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5318034000 tx=0x7f531802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 msgr2=0x7f53281e7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f531000c430 tx=0x7f531000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 shutdown_connections 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f530406c920 0x7f530406ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f532815d9c0 0x7f53280ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532815a7e0 0x7f53281e7540 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 --2- 172.21.15.43:0/2884471176 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53280b5410 0x7f53281e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 >> 172.21.15.43:0/2884471176 conn(0x7f5328068090 msgr2=0x7f53280cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 shutdown_connections 2024-04-26T20:27:19.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:19.909+0000 7f5327e00640 1 -- 172.21.15.43:0/2884471176 wait complete. 2024-04-26T20:27:19.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:20.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.136+0000 7fd3b7200640 1 Processor -- start 2024-04-26T20:27:20.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.136+0000 7fd3b7200640 1 -- start start 2024-04-26T20:27:20.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 0x7fd3b80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8159840 0x7fd3b80b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3b8160260 0x7fd3b80b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80b65e0 con 0x7fd3b8160260 2024-04-26T20:27:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80b28e0 con 0x7fd3b8159460 2024-04-26T20:27:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b8074320 con 0x7fd3b8159840 2024-04-26T20:27:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3b8160260 0x7fd3b80b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3b8160260 0x7fd3b80b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41326/0 (socket says 172.21.15.43:41326) 2024-04-26T20:27:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 -- 172.21.15.43:0/985833189 learned_addr learned my addr 172.21.15.43:0/985833189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b5e00640 1 --2- 172.21.15.43:0/985833189 >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 0x7fd3b80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 -- 172.21.15.43:0/985833189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8159840 msgr2=0x7fd3b80b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 --2- 172.21.15.43:0/985833189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8159840 0x7fd3b80b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 -- 172.21.15.43:0/985833189 >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 msgr2=0x7fd3b80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 --2- 172.21.15.43:0/985833189 >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 0x7fd3b80b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.138+0000 7fd3b6800640 1 -- 172.21.15.43:0/985833189 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b815f780 con 0x7fd3b8160260 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.139+0000 7fd3b5e00640 1 --2- 172.21.15.43:0/985833189 >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 0x7fd3b80b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b6800640 1 --2- 172.21.15.43:0/985833189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b8160260 0x7fd3b80b60a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8072f20 tx=0x7fd3b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3025d71a480371d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b4a00640 1 -- 172.21.15.43:0/985833189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0036070 con 0x7fd3b8160260 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b4a00640 1 -- 172.21.15.43:0/985833189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0031600 con 0x7fd3b8160260 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b4a00640 1 -- 172.21.15.43:0/985833189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00319d0 con 0x7fd3b8160260 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b7200640 1 -- 172.21.15.43:0/985833189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b8160260 msgr2=0x7fd3b80b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.140+0000 7fd3b7200640 1 --2- 172.21.15.43:0/985833189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b8160260 0x7fd3b80b60a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8072f20 tx=0x7fd3b002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 -- 172.21.15.43:0/985833189 shutdown_connections 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 --2- 172.21.15.43:0/985833189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b8160260 0x7fd3b80b60a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 --2- 172.21.15.43:0/985833189 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8159840 0x7fd3b80b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 --2- 172.21.15.43:0/985833189 >> v2:172.21.15.164:3300/0 conn(0x7fd3b8159460 0x7fd3b80b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 -- 172.21.15.43:0/985833189 >> 172.21.15.43:0/985833189 conn(0x7fd3b80729c0 msgr2=0x7fd3b8073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 -- 172.21.15.43:0/985833189 shutdown_connections 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.141+0000 7fd3b7200640 1 -- 172.21.15.43:0/985833189 wait complete. 2024-04-26T20:27:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.142+0000 7fd3b7200640 1 Processor -- start 2024-04-26T20:27:20.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.142+0000 7fd3b7200640 1 -- start start 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b8159460 0x7fd3b809ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8160260 0x7fd3b8148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b80b4610 con 0x7fd3b814b650 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b815fd40 con 0x7fd3b8159460 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.143+0000 7fd3b7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b8073f80 con 0x7fd3b8160260 2024-04-26T20:27:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41336/0 (socket says 172.21.15.43:41336) 2024-04-26T20:27:20.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 -- 172.21.15.43:0/1871038412 learned_addr learned my addr 172.21.15.43:0/1871038412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b5e00640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b8159460 0x7fd3b809ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b5400640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8160260 0x7fd3b8148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 -- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8160260 msgr2=0x7fd3b8148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8160260 0x7fd3b8148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 -- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b8159460 msgr2=0x7fd3b809ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b8159460 0x7fd3b809ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.144+0000 7fd3b6800640 1 -- 172.21.15.43:0/1871038412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b8149860 con 0x7fd3b814b650 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3b6800640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009f40 tx=0x7fd3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0036050 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0042070 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0009510 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b8149b40 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.145+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b814a100 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.146+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b80506c0 con 0x7fd3b814b650 2024-04-26T20:27:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.147+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3b0007bf0 con 0x7fd3b814b650 2024-04-26T20:27:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.149+0000 7fd3ae000640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.149+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd3b00acfb0 con 0x7fd3b814b650 2024-04-26T20:27:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.149+0000 7fd3b5e00640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.150+0000 7fd3b5e00640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800a990 tx=0x7fd3a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.153+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3b007af10 con 0x7fd3b814b650 2024-04-26T20:27:20.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.353+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3b80d16d0 con 0x7fd3b814b650 2024-04-26T20:27:20.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.354+0000 7fd3ae000640 1 -- 172.21.15.43:0/1871038412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd3b007a810 con 0x7fd3b814b650 2024-04-26T20:27:20.355 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 msgr2=0x7fd39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800a990 tx=0x7fd3a8008040 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 msgr2=0x7fd3b815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009f40 tx=0x7fd3b0004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 shutdown_connections 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3b8160260 0x7fd3b8148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3b814b650 0x7fd3b815ada0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 --2- 172.21.15.43:0/1871038412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3b8159460 0x7fd3b809ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 >> 172.21.15.43:0/1871038412 conn(0x7fd3b80729c0 msgr2=0x7fd3b8162af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 shutdown_connections 2024-04-26T20:27:20.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.357+0000 7fd3b7200640 1 -- 172.21.15.43:0/1871038412 wait complete. 2024-04-26T20:27:20.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.584+0000 7f84fe800640 1 Processor -- start 2024-04-26T20:27:20.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.584+0000 7f84fe800640 1 -- start start 2024-04-26T20:27:20.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.585+0000 7f84fe800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84f815f180 0x7f84f80d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.585+0000 7f84fe800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.585+0000 7f84fe800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f84f80cbff0 0x7f84f80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.585+0000 7f84fe800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f84f80d1340 con 0x7f84f815f180 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fe800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f84f80d0490 con 0x7f84f80cbff0 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fe800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80d0730 con 0x7f84f80d57d0 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84f815f180 0x7f84f80d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84f815f180 0x7f84f80d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41340/0 (socket says 172.21.15.43:41340) 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 -- 172.21.15.43:0/2281659558 learned_addr learned my addr 172.21.15.43:0/2281659558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 -- 172.21.15.43:0/2281659558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 msgr2=0x7f84f80d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fca00640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fde00640 1 --2- 172.21.15.43:0/2281659558 >> v2:172.21.15.164:3300/0 conn(0x7f84f80cbff0 0x7f84f80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 -- 172.21.15.43:0/2281659558 >> v2:172.21.15.164:3300/0 conn(0x7f84f80cbff0 msgr2=0x7f84f80cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 --2- 172.21.15.43:0/2281659558 >> v2:172.21.15.164:3300/0 conn(0x7f84f80cbff0 0x7f84f80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fd400640 1 -- 172.21.15.43:0/2281659558 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f80d0a10 con 0x7f84f815f180 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.586+0000 7f84fca00640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.587+0000 7f84fd400640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815f180 0x7f84f80d15c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f80c8130 tx=0x7f84ec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ae5455e392708 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84f7e00640 1 -- 172.21.15.43:0/2281659558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec037070 con 0x7f84f815f180 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84f7e00640 1 -- 172.21.15.43:0/2281659558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ec032440 con 0x7f84f815f180 2024-04-26T20:27:20.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84f7e00640 1 -- 172.21.15.43:0/2281659558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec032810 con 0x7f84f815f180 2024-04-26T20:27:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84fe800640 1 -- 172.21.15.43:0/2281659558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815f180 msgr2=0x7f84f80d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84fe800640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815f180 0x7f84f80d15c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f80c8130 tx=0x7f84ec02a140 comp rx=0 tx=0).stop 2024-04-26T20:27:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.588+0000 7f84fe800640 1 -- 172.21.15.43:0/2281659558 shutdown_connections 2024-04-26T20:27:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 --2- 172.21.15.43:0/2281659558 >> v2:172.21.15.164:3300/0 conn(0x7f84f80cbff0 0x7f84f80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 --2- 172.21.15.43:0/2281659558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f815f180 0x7f84f80d15c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 -- 172.21.15.43:0/2281659558 >> 172.21.15.43:0/2281659558 conn(0x7f84f80d4bb0 msgr2=0x7f84f80d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 -- 172.21.15.43:0/2281659558 shutdown_connections 2024-04-26T20:27:20.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.589+0000 7f84fe800640 1 -- 172.21.15.43:0/2281659558 wait complete. 2024-04-26T20:27:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.590+0000 7f84fe800640 1 Processor -- start 2024-04-26T20:27:20.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.591+0000 7f84fe800640 1 -- start start 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f815f180 0x7f84f815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80cc9c0 con 0x7f84f80cbff0 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80d0fd0 con 0x7f84f815f180 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fe800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80ccb40 con 0x7f84f80d57d0 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41348/0 (socket says 172.21.15.43:41348) 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 -- 172.21.15.43:0/4099356736 learned_addr learned my addr 172.21.15.43:0/4099356736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fca00640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fde00640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f815f180 0x7f84f815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 -- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 msgr2=0x7f84f80c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 -- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f815f180 msgr2=0x7f84f815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f815f180 0x7f84f815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.592+0000 7f84fd400640 1 -- 172.21.15.43:0/4099356736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f81f4f20 con 0x7f84f80cbff0 2024-04-26T20:27:20.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84fd400640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84ec02a180 tx=0x7f84ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec037040 con 0x7f84f80cbff0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ec042070 con 0x7f84f80cbff0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f81f5200 con 0x7f84f80cbff0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec030bb0 con 0x7f84f80cbff0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.593+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f80c86e0 con 0x7f84f80cbff0 2024-04-26T20:27:20.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.595+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f80506c0 con 0x7f84f80cbff0 2024-04-26T20:27:20.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.595+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84ec047460 con 0x7f84f80cbff0 2024-04-26T20:27:20.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.596+0000 7f84f5600640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 0x7f84d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:20.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.596+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f84ec0acc00 con 0x7f84f80cbff0 2024-04-26T20:27:20.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.597+0000 7f84fca00640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 0x7f84d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:20.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.597+0000 7f84fca00640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 0x7f84d806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84f80b7980 tx=0x7f84e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:20.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.601+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84ec045360 con 0x7f84f80cbff0 2024-04-26T20:27:20.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.801+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f84f815e6e0 con 0x7f84f80cbff0 2024-04-26T20:27:20.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.803+0000 7f84f5600640 1 -- 172.21.15.43:0/4099356736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f84ec07abf0 con 0x7f84f80cbff0 2024-04-26T20:27:20.803 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.805+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 msgr2=0x7f84d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.805+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 0x7f84d806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84f80b7980 tx=0x7f84e4008040 comp rx=0 tx=0).stop 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.805+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 msgr2=0x7f84f80ba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.805+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84ec02a180 tx=0x7f84ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 shutdown_connections 2024-04-26T20:27:20.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84d806c8f0 0x7f84d806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84f80d57d0 0x7f84f80c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84f815f180 0x7f84f815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 --2- 172.21.15.43:0/4099356736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84f80cbff0 0x7f84f80ba050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 >> 172.21.15.43:0/4099356736 conn(0x7f84f80d4bb0 msgr2=0x7f84f80d3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 shutdown_connections 2024-04-26T20:27:20.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:20.806+0000 7f84fe800640 1 -- 172.21.15.43:0/4099356736 wait complete. 2024-04-26T20:27:20.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:21.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.030+0000 7fd0b6600640 1 Processor -- start 2024-04-26T20:27:21.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.030+0000 7fd0b6600640 1 -- start start 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.032+0000 7fd0b6600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd0b015f3f0 0x7fd0b00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.032+0000 7fd0b6600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd0b00b5940 0x7fd0b0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.032+0000 7fd0b6600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b015f920 con 0x7fd0b0155cd0 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.032+0000 7fd0b6600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b015fc00 con 0x7fd0b015f3f0 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.032+0000 7fd0b6600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b015fee0 con 0x7fd0b00b5940 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0b5200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd0b0155cd0 0x7fd0b0072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58194/0 (socket says 172.21.15.43:58194) 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0afe00640 1 --2- 172.21.15.43:0/1646081446 >> v2:172.21.15.43:3300/0 conn(0x7fd0b015f3f0 0x7fd0b00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0b5200640 1 -- 172.21.15.43:0/1646081446 learned_addr learned my addr 172.21.15.43:0/1646081446 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:21.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0b5c00640 1 --2- 172.21.15.43:0/1646081446 >> v2:172.21.15.164:3300/0 conn(0x7fd0b00b5940 0x7fd0b0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0afe00640 1 -- 172.21.15.43:0/1646081446 >> v2:172.21.15.164:3300/0 conn(0x7fd0b00b5940 msgr2=0x7fd0b0162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:21.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0afe00640 1 --2- 172.21.15.43:0/1646081446 >> v2:172.21.15.164:3300/0 conn(0x7fd0b00b5940 0x7fd0b0162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0afe00640 1 -- 172.21.15.43:0/1646081446 >> v1:172.21.15.43:6789/0 conn(0x7fd0b0155cd0 legacy=0x7fd0b0072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0afe00640 1 -- 172.21.15.43:0/1646081446 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b01601c0 con 0x7fd0b015f3f0 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.033+0000 7fd0af400640 1 -- 172.21.15.43:0/1646081446 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3105210519 0 0) 0x7fd0b015f920 con 0x7fd0b0155cd0 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.034+0000 7fd0afe00640 1 --2- 172.21.15.43:0/1646081446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b00b5400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd0b015cf30 tx=0x7fd09c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce4ade79f1acc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0af400640 1 -- 172.21.15.43:0/1646081446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c037070 con 0x7fd0b015f3f0 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0af400640 1 -- 172.21.15.43:0/1646081446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c032440 con 0x7fd0b015f3f0 2024-04-26T20:27:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0af400640 1 -- 172.21.15.43:0/1646081446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c032810 con 0x7fd0b015f3f0 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0b6600640 1 -- 172.21.15.43:0/1646081446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 msgr2=0x7fd0b00b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1646081446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b00b5400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd0b015cf30 tx=0x7fd09c02a250 comp rx=0 tx=0).stop 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0b6600640 1 -- 172.21.15.43:0/1646081446 shutdown_connections 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.035+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1646081446 >> v2:172.21.15.164:3300/0 conn(0x7fd0b00b5940 0x7fd0b0162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.036+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1646081446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b00b5400 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.036+0000 7fd0b6600640 1 -- 172.21.15.43:0/1646081446 >> 172.21.15.43:0/1646081446 conn(0x7fd0b0068090 msgr2=0x7fd0b00684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.036+0000 7fd0b6600640 1 -- 172.21.15.43:0/1646081446 shutdown_connections 2024-04-26T20:27:21.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.036+0000 7fd0b6600640 1 -- 172.21.15.43:0/1646081446 wait complete. 2024-04-26T20:27:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.037+0000 7fd0b6600640 1 Processor -- start 2024-04-26T20:27:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.037+0000 7fd0b6600640 1 -- start start 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b00b5940 0x7fd0b015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd0b0155cd0 0x7fd0b0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b01560b0 con 0x7fd0b015f3f0 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0072d10 con 0x7fd0b00b5940 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.038+0000 7fd0b6600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b0156230 con 0x7fd0b0155cd0 2024-04-26T20:27:21.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41362/0 (socket says 172.21.15.43:41362) 2024-04-26T20:27:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 -- 172.21.15.43:0/1162276142 learned_addr learned my addr 172.21.15.43:0/1162276142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 -- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd0b0155cd0 msgr2=0x7fd0b0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd0b0155cd0 0x7fd0b0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 -- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b00b5940 msgr2=0x7fd0b015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b00b5940 0x7fd0b015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 -- 172.21.15.43:0/1162276142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b01f4ab0 con 0x7fd0b015f3f0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.039+0000 7fd0b5c00640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8007e40 tx=0x7fd0a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.040+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8014620 con 0x7fd0b015f3f0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.040+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a8019070 con 0x7fd0b015f3f0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.040+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8014a20 con 0x7fd0b015f3f0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.040+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b01f4d90 con 0x7fd0b015f3f0 2024-04-26T20:27:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.040+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b01f5350 con 0x7fd0b015f3f0 2024-04-26T20:27:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.041+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b015c960 con 0x7fd0b015f3f0 2024-04-26T20:27:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.042+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0a8014bc0 con 0x7fd0b015f3f0 2024-04-26T20:27:21.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.043+0000 7fd0acc00640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 0x7fd09006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.043+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd0a808c850 con 0x7fd0b015f3f0 2024-04-26T20:27:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.044+0000 7fd0b5200640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 0x7fd09006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.044+0000 7fd0b5200640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 0x7fd09006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4007fd0 tx=0x7fd0a4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.048+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0a805a840 con 0x7fd0b015f3f0 2024-04-26T20:27:21.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.249+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0b00506c0 con 0x7fd0b015f3f0 2024-04-26T20:27:21.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.250+0000 7fd0acc00640 1 -- 172.21.15.43:0/1162276142 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd0a805a140 con 0x7fd0b015f3f0 2024-04-26T20:27:21.250 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.252+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 msgr2=0x7fd09006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.252+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 0x7fd09006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4007fd0 tx=0x7fd0a4007920 comp rx=0 tx=0).stop 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.252+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 msgr2=0x7fd0b0142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.252+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8007e40 tx=0x7fd0a800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 shutdown_connections 2024-04-26T20:27:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd09006c8f0 0x7fd09006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd0b0155cd0 0x7fd0b0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b015f3f0 0x7fd0b0142530 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 --2- 172.21.15.43:0/1162276142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0b00b5940 0x7fd0b015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 >> 172.21.15.43:0/1162276142 conn(0x7fd0b0068090 msgr2=0x7fd0b0068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 shutdown_connections 2024-04-26T20:27:21.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.253+0000 7fd0b6600640 1 -- 172.21.15.43:0/1162276142 wait complete. 2024-04-26T20:27:21.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.496+0000 7f1d97200640 1 Processor -- start 2024-04-26T20:27:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.496+0000 7f1d97200640 1 -- start start 2024-04-26T20:27:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d97200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1d9815bb90 0x7f1d980737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d97200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d980afb70 0x7f1d9815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d97200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d98072db0 con 0x7f1d980af710 2024-04-26T20:27:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d97200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980cec10 con 0x7f1d980afb70 2024-04-26T20:27:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d97200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980725a0 con 0x7f1d9815bb90 2024-04-26T20:27:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d95400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1d980af710 0x7f1d9815f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58196/0 (socket says 172.21.15.43:58196) 2024-04-26T20:27:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d95400640 1 -- 172.21.15.43:0/2706980078 learned_addr learned my addr 172.21.15.43:0/2706980078 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d96800640 1 --2- 172.21.15.43:0/2706980078 >> v2:172.21.15.43:3300/0 conn(0x7f1d980afb70 0x7f1d9815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.498+0000 7f1d95e00640 1 --2- 172.21.15.43:0/2706980078 >> v2:172.21.15.164:3300/0 conn(0x7f1d9815bb90 0x7f1d980737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d94a00640 1 -- 172.21.15.43:0/2706980078 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199251755 0 0) 0x7f1d98072db0 con 0x7f1d980af710 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d94a00640 1 -- 172.21.15.43:0/2706980078 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d7c003520 con 0x7f1d980af710 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d96800640 1 -- 172.21.15.43:0/2706980078 >> v2:172.21.15.164:3300/0 conn(0x7f1d9815bb90 msgr2=0x7f1d980737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d96800640 1 --2- 172.21.15.43:0/2706980078 >> v2:172.21.15.164:3300/0 conn(0x7f1d9815bb90 0x7f1d980737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d96800640 1 -- 172.21.15.43:0/2706980078 >> v1:172.21.15.43:6789/0 conn(0x7f1d980af710 legacy=0x7f1d9815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.499+0000 7f1d96800640 1 -- 172.21.15.43:0/2706980078 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980cee40 con 0x7f1d980afb70 2024-04-26T20:27:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.500+0000 7f1d96800640 1 --2- 172.21.15.43:0/2706980078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9815fd20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806f820 tx=0x7f1d9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa39f2677bcedb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.500+0000 7f1d94a00640 1 -- 172.21.15.43:0/2706980078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90037070 con 0x7f1d980afb70 2024-04-26T20:27:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.500+0000 7f1d94a00640 1 -- 172.21.15.43:0/2706980078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d90032440 con 0x7f1d980afb70 2024-04-26T20:27:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.500+0000 7f1d94a00640 1 -- 172.21.15.43:0/2706980078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90032810 con 0x7f1d980afb70 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 -- 172.21.15.43:0/2706980078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 msgr2=0x7f1d9815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 --2- 172.21.15.43:0/2706980078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9815fd20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d9806f820 tx=0x7f1d9002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 -- 172.21.15.43:0/2706980078 shutdown_connections 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 --2- 172.21.15.43:0/2706980078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9815fd20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 --2- 172.21.15.43:0/2706980078 >> v2:172.21.15.164:3300/0 conn(0x7f1d9815bb90 0x7f1d980737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 -- 172.21.15.43:0/2706980078 >> 172.21.15.43:0/2706980078 conn(0x7f1d98068090 msgr2=0x7f1d98068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 -- 172.21.15.43:0/2706980078 shutdown_connections 2024-04-26T20:27:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.501+0000 7f1d97200640 1 -- 172.21.15.43:0/2706980078 wait complete. 2024-04-26T20:27:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.503+0000 7f1d97200640 1 Processor -- start 2024-04-26T20:27:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.504+0000 7f1d97200640 1 -- start start 2024-04-26T20:27:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d980af710 0x7f1d981479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d9815bb90 0x7f1d9814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98072ad0 con 0x7f1d980afb70 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d980d0610 con 0x7f1d980af710 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d97200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980cf0e0 con 0x7f1d9815bb90 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d95400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d95400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41376/0 (socket says 172.21.15.43:41376) 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d95400640 1 -- 172.21.15.43:0/1601829993 learned_addr learned my addr 172.21.15.43:0/1601829993 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d95e00640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d980af710 0x7f1d981479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.505+0000 7f1d96800640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d9815bb90 0x7f1d9814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 -- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d9815bb90 msgr2=0x7f1d9814b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d9815bb90 0x7f1d9814b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 -- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d980af710 msgr2=0x7f1d981479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d980af710 0x7f1d981479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 -- 172.21.15.43:0/1601829993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d98145c40 con 0x7f1d980afb70 2024-04-26T20:27:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d95400640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800cb70 tx=0x7f1d8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88017070 con 0x7f1d980afb70 2024-04-26T20:27:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8800ad40 con 0x7f1d980afb70 2024-04-26T20:27:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88012620 con 0x7f1d980afb70 2024-04-26T20:27:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.506+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d98145f20 con 0x7f1d980afb70 2024-04-26T20:27:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.507+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981466f0 con 0x7f1d980afb70 2024-04-26T20:27:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.508+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98069350 con 0x7f1d980afb70 2024-04-26T20:27:21.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.509+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8801a070 con 0x7f1d980afb70 2024-04-26T20:27:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.510+0000 7f1d8e000640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 0x7f1d7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.510+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f1d8808bd60 con 0x7f1d980afb70 2024-04-26T20:27:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.511+0000 7f1d95e00640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 0x7f1d7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.511+0000 7f1d95e00640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 0x7f1d7806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009a30 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.515+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d88059df0 con 0x7f1d980afb70 2024-04-26T20:27:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.733+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d9815f380 con 0x7f1d980afb70 2024-04-26T20:27:21.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.734+0000 7f1d8e000640 1 -- 172.21.15.43:0/1601829993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1d880596f0 con 0x7f1d980afb70 2024-04-26T20:27:21.735 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 msgr2=0x7f1d7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 0x7f1d7806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009a30 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 msgr2=0x7f1d9814b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800cb70 tx=0x7f1d8800a430 comp rx=0 tx=0).stop 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 shutdown_connections 2024-04-26T20:27:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1d7806c8f0 0x7f1d7806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d9815bb90 0x7f1d9814b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d980afb70 0x7f1d9814b3d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 --2- 172.21.15.43:0/1601829993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d980af710 0x7f1d981479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 >> 172.21.15.43:0/1601829993 conn(0x7f1d98068090 msgr2=0x7f1d980b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.737+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 shutdown_connections 2024-04-26T20:27:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:21.738+0000 7f1d97200640 1 -- 172.21.15.43:0/1601829993 wait complete. 2024-04-26T20:27:21.760 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:27:21.760 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:21.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.025+0000 7fab97000640 1 Processor -- start 2024-04-26T20:27:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.026+0000 7fab97000640 1 -- start start 2024-04-26T20:27:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.027+0000 7fab97000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab900d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab97000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab900d3810 0x7fab900b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab97000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab900d0480 con 0x7fab900d1a00 2024-04-26T20:27:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab97000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fab900d0ce0 con 0x7fab900d3810 2024-04-26T20:27:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab97000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab900d0720 con 0x7fab900d1620 2024-04-26T20:27:22.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab95200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fab900d1a00 0x7fab900b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58208/0 (socket says 172.21.15.43:58208) 2024-04-26T20:27:22.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab95200640 1 -- 172.21.15.43:0/3478252836 learned_addr learned my addr 172.21.15.43:0/3478252836 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010224112 0 0) 0x7fab900d0480 con 0x7fab900d1a00 2024-04-26T20:27:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.028+0000 7fab95c00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab900d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab78003520 con 0x7fab900d1a00 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab96600640 1 --2- 172.21.15.43:0/3478252836 >> v2:172.21.15.164:3300/0 conn(0x7fab900d3810 0x7fab900b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1862320015 0 0) 0x7fab78003520 con 0x7fab900d1a00 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 msgr2=0x7fab900d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab900d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 >> v2:172.21.15.164:3300/0 conn(0x7fab900d3810 msgr2=0x7fab900b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 --2- 172.21.15.43:0/3478252836 >> v2:172.21.15.164:3300/0 conn(0x7fab900d3810 0x7fab900b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900d0a00 con 0x7fab900d1a00 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fab7c001670 con 0x7fab900d1a00 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 >> v1:172.21.15.43:6789/0 conn(0x7fab900d1a00 legacy=0x7fab900b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab900d0ce0 con 0x7fab78004d40 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab900d0720 con 0x7fab780039b0 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab7800bb20 con 0x7fab900d1620 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.029+0000 7fab96600640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab78004d40 0x7fab780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab95c00640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab95200640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab96600640 1 -- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 msgr2=0x7fab78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab96600640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab96600640 1 -- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab780039b0 msgr2=0x7fab78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab96600640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab780039b0 0x7fab78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab96600640 1 -- 172.21.15.43:0/3478252836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab78003520 con 0x7fab78004d40 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.030+0000 7fab95200640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab900d1620 0x7fab78004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.031+0000 7fab96600640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab8800ba10 tx=0x7fab78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495dfea6ce7c217b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.031+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88030070 con 0x7fab78004d40 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.031+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8802ae20 con 0x7fab78004d40 2024-04-26T20:27:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.031+0000 7fab8fe00640 1 -- 172.21.15.43:0/3478252836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8800b5e0 con 0x7fab78004d40 2024-04-26T20:27:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 -- 172.21.15.43:0/3478252836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab78004d40 msgr2=0x7fab780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 --2- 172.21.15.43:0/3478252836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab78004d40 0x7fab780051c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fab8800ba10 tx=0x7fab78005830 comp rx=0 tx=0).stop 2024-04-26T20:27:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab96600640 1 -- 172.21.15.43:0/3478252836 reap_dead start 2024-04-26T20:27:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 -- 172.21.15.43:0/3478252836 shutdown_connections 2024-04-26T20:27:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 -- 172.21.15.43:0/3478252836 >> 172.21.15.43:0/3478252836 conn(0x7fab900d4c30 msgr2=0x7fab900d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 -- 172.21.15.43:0/3478252836 shutdown_connections 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.032+0000 7fab97000640 1 -- 172.21.15.43:0/3478252836 wait complete. 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.034+0000 7fab97000640 1 Processor -- start 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.034+0000 7fab97000640 1 -- start start 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab78004d40 0x7fab900ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab9014bf90 0x7fab900b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab900d5750 con 0x7fab9014bb10 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab900d1e60 con 0x7fab78004d40 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab97000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab900d58d0 con 0x7fab9014bf90 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab95200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab95200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41402/0 (socket says 172.21.15.43:41402) 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.035+0000 7fab95200640 1 -- 172.21.15.43:0/3719190244 learned_addr learned my addr 172.21.15.43:0/3719190244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab96600640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab9014bf90 0x7fab900b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95c00640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab78004d40 0x7fab900ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 -- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab9014bf90 msgr2=0x7fab900b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab9014bf90 0x7fab900b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 -- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab78004d40 msgr2=0x7fab900ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab78004d40 0x7fab900ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 -- 172.21.15.43:0/3719190244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab900cd740 con 0x7fab9014bb10 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.036+0000 7fab95200640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00db10 tx=0x7fab7c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.037+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c019070 con 0x7fab9014bb10 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.037+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c0074e0 con 0x7fab9014bb10 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.037+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0047e0 con 0x7fab9014bb10 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.037+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab900b8ff0 con 0x7fab9014bb10 2024-04-26T20:27:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.037+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab900b95b0 con 0x7fab9014bb10 2024-04-26T20:27:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.038+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab90069350 con 0x7fab9014bb10 2024-04-26T20:27:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.039+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab7c004980 con 0x7fab9014bb10 2024-04-26T20:27:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.040+0000 7fab8d600640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.040+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fab7c08bae0 con 0x7fab9014bb10 2024-04-26T20:27:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.040+0000 7fab95c00640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.041+0000 7fab95c00640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 0x7fab7006ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab840041e0 tx=0x7fab8400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.044+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab900cd740 con 0x7fab9014bb10 2024-04-26T20:27:22.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.253+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab9006dce0 con 0x7fab9014bb10 2024-04-26T20:27:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.256+0000 7fab8d600640 1 -- 172.21.15.43:0/3719190244 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fab7c0599b0 con 0x7fab9014bb10 2024-04-26T20:27:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:22.257 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 msgr2=0x7fab7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 0x7fab7006ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab840041e0 tx=0x7fab8400b040 comp rx=0 tx=0).stop 2024-04-26T20:27:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 msgr2=0x7fab900b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00db10 tx=0x7fab7c007e10 comp rx=0 tx=0).stop 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 shutdown_connections 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab9014bf90 0x7fab900b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab9014bb10 0x7fab900b62a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 --2- 172.21.15.43:0/3719190244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab78004d40 0x7fab900ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 >> 172.21.15.43:0/3719190244 conn(0x7fab900d4c30 msgr2=0x7fab900cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 shutdown_connections 2024-04-26T20:27:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.259+0000 7fab97000640 1 -- 172.21.15.43:0/3719190244 wait complete. 2024-04-26T20:27:22.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:22.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:22.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.502+0000 7f9a13200640 1 Processor -- start 2024-04-26T20:27:22.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.502+0000 7f9a13200640 1 -- start start 2024-04-26T20:27:22.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a13200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a140b63e0 0x7f9a140af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a13200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a140ae220 con 0x7f9a1415d250 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a13200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a14073f80 con 0x7f9a14075ba0 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a13200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9a14074260 con 0x7f9a140b63e0 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a11e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9a1415d250 0x7f9a1415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58218/0 (socket says 172.21.15.43:58218) 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a11e00640 1 -- 172.21.15.43:0/2269671281 learned_addr learned my addr 172.21.15.43:0/2269671281 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.504+0000 7f9a11400640 1 --2- 172.21.15.43:0/2269671281 >> v2:172.21.15.43:3300/0 conn(0x7f9a140b63e0 0x7f9a140af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.505+0000 7f9a10a00640 1 -- 172.21.15.43:0/2269671281 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893242033 0 0) 0x7f9a04001110 con 0x7f9a1415d250 2024-04-26T20:27:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.505+0000 7f9a11400640 1 -- 172.21.15.43:0/2269671281 >> v1:172.21.15.164:6789/0 conn(0x7f9a14075ba0 legacy=0x7f9a1415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.505+0000 7f9a11400640 1 -- 172.21.15.43:0/2269671281 >> v1:172.21.15.43:6789/0 conn(0x7f9a1415d250 legacy=0x7f9a1415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.505+0000 7f9a11400640 1 -- 172.21.15.43:0/2269671281 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1415ed10 con 0x7f9a140b63e0 2024-04-26T20:27:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.506+0000 7f9a11400640 1 --2- 172.21.15.43:0/2269671281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a140b63e0 0x7f9a140af750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a000097a0 tx=0x7f9a0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae9ab4bfa341749 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.506+0000 7f9a10a00640 1 -- 172.21.15.43:0/2269671281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00037070 con 0x7f9a140b63e0 2024-04-26T20:27:22.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.506+0000 7f9a10a00640 1 -- 172.21.15.43:0/2269671281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00032440 con 0x7f9a140b63e0 2024-04-26T20:27:22.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.506+0000 7f9a10a00640 1 -- 172.21.15.43:0/2269671281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00032810 con 0x7f9a140b63e0 2024-04-26T20:27:22.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 -- 172.21.15.43:0/2269671281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a140b63e0 msgr2=0x7f9a140af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 --2- 172.21.15.43:0/2269671281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a140b63e0 0x7f9a140af750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9a000097a0 tx=0x7f9a0002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 -- 172.21.15.43:0/2269671281 shutdown_connections 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 --2- 172.21.15.43:0/2269671281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a140b63e0 0x7f9a140af750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 -- 172.21.15.43:0/2269671281 >> 172.21.15.43:0/2269671281 conn(0x7f9a140729c0 msgr2=0x7f9a14073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 -- 172.21.15.43:0/2269671281 shutdown_connections 2024-04-26T20:27:22.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.507+0000 7f9a13200640 1 -- 172.21.15.43:0/2269671281 wait complete. 2024-04-26T20:27:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.509+0000 7f9a13200640 1 Processor -- start 2024-04-26T20:27:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.509+0000 7f9a13200640 1 -- start start 2024-04-26T20:27:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a140b63e0 0x7f9a14153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 0x7f9a1409a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14074540 con 0x7f9a14075ba0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1415f2d0 con 0x7f9a140b63e0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a13200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1415dd40 con 0x7f9a1415d250 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a11e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.510+0000 7f9a11e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41426/0 (socket says 172.21.15.43:41426) 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 -- 172.21.15.43:0/2149127709 learned_addr learned my addr 172.21.15.43:0/2149127709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11400640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a140b63e0 0x7f9a14153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a12800640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 0x7f9a1409a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 -- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 msgr2=0x7f9a1409a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 0x7f9a1409a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 -- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a140b63e0 msgr2=0x7f9a14153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a140b63e0 0x7f9a14153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 -- 172.21.15.43:0/2149127709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a14154430 con 0x7f9a14075ba0 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a12800640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 0x7f9a1409a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:22.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.511+0000 7f9a11e00640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c430 tx=0x7f9a0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.512+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040074e0 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.512+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a040047e0 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.512+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04004bd0 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.512+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a14154710 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.512+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a14154cd0 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.513+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a140506c0 con 0x7f9a14075ba0 2024-04-26T20:27:22.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.514+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a04011070 con 0x7f9a14075ba0 2024-04-26T20:27:22.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.515+0000 7f9a0a000640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.515+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f9a0408bce0 con 0x7f9a14075ba0 2024-04-26T20:27:22.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.515+0000 7f9a11400640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.516+0000 7f9a11400640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 0x7f99f406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034000 tx=0x7f9a0002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.520+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a140ae220 con 0x7f9a14075ba0 2024-04-26T20:27:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.725+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a14159390 con 0x7f9a14075ba0 2024-04-26T20:27:22.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.726+0000 7f9a0a000640 1 -- 172.21.15.43:0/2149127709 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9a04059c40 con 0x7f9a14075ba0 2024-04-26T20:27:22.726 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 msgr2=0x7f99f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 0x7f99f406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a00034000 tx=0x7f9a0002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 msgr2=0x7f9a14153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c430 tx=0x7f9a0400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 shutdown_connections 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a1415d250 0x7f9a1409a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a140b63e0 0x7f9a14153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 --2- 172.21.15.43:0/2149127709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a14075ba0 0x7f9a14153e50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 >> 172.21.15.43:0/2149127709 conn(0x7f9a140729c0 msgr2=0x7f9a140b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 shutdown_connections 2024-04-26T20:27:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.728+0000 7f9a13200640 1 -- 172.21.15.43:0/2149127709 wait complete. 2024-04-26T20:27:22.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:22.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.965+0000 7f3f4da00640 1 Processor -- start 2024-04-26T20:27:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.965+0000 7f3f4da00640 1 -- start start 2024-04-26T20:27:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4da00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3f4815a7e0 0x7f3f480b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4da00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f480b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4da00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f480b3900 con 0x7f3f4815d2b0 2024-04-26T20:27:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4da00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4815abc0 con 0x7f3f4815a7e0 2024-04-26T20:27:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4da00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480b3b70 con 0x7f3f480b5410 2024-04-26T20:27:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4d000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3f4815d2b0 0x7f3f4815d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58230/0 (socket says 172.21.15.43:58230) 2024-04-26T20:27:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.967+0000 7f3f4d000640 1 -- 172.21.15.43:0/2093453870 learned_addr learned my addr 172.21.15.43:0/2093453870 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f47e00640 1 --2- 172.21.15.43:0/2093453870 >> v2:172.21.15.164:3300/0 conn(0x7f3f4815a7e0 0x7f3f480b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f47400640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f480b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4105638465 0 0) 0x7f3f480b3900 con 0x7f3f4815d2b0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c003520 con 0x7f3f4815d2b0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2783056901 0 0) 0x7f3f2c003520 con 0x7f3f4815d2b0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 msgr2=0x7f3f480b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f480b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 >> v2:172.21.15.164:3300/0 conn(0x7f3f4815a7e0 msgr2=0x7f3f480b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 --2- 172.21.15.43:0/2093453870 >> v2:172.21.15.164:3300/0 conn(0x7f3f4815a7e0 0x7f3f480b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480b3e50 con 0x7f3f4815d2b0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f3f400019f0 con 0x7f3f4815d2b0 2024-04-26T20:27:22.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 >> v1:172.21.15.43:6789/0 conn(0x7f3f4815d2b0 legacy=0x7f3f4815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f2c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.968+0000 7f3f46a00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c003fa0 0x7f3f2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f46a00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4815abc0 con 0x7f3f2c004d40 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480b3b70 con 0x7f3f2c003fa0 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c00b9f0 con 0x7f3f480b5410 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f4d000640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f47e00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f2c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.969+0000 7f3f47400640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c003fa0 0x7f3f2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.970+0000 7f3f4d000640 1 -- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 msgr2=0x7f3f2c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.970+0000 7f3f4d000640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f480b5410 0x7f3f2c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.970+0000 7f3f4d000640 1 -- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c003fa0 msgr2=0x7f3f2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.970+0000 7f3f4d000640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c003fa0 0x7f3f2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.970+0000 7f3f4d000640 1 -- 172.21.15.43:0/2093453870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c003520 con 0x7f3f2c004d40 2024-04-26T20:27:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f4d000640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000a920 tx=0x7f3f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74612a6c5ccd887c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000a4c0 con 0x7f3f2c004d40 2024-04-26T20:27:22.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4002f070 con 0x7f3f2c004d40 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f46a00640 1 -- 172.21.15.43:0/2093453870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4002bd40 con 0x7f3f2c004d40 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f4da00640 1 -- 172.21.15.43:0/2093453870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f2c004d40 msgr2=0x7f3f2c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f4da00640 1 --2- 172.21.15.43:0/2093453870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f2c004d40 0x7f3f2c0051c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000a920 tx=0x7f3f2c005830 comp rx=0 tx=0).stop 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.971+0000 7f3f4d000640 1 -- 172.21.15.43:0/2093453870 reap_dead start 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.972+0000 7f3f4da00640 1 -- 172.21.15.43:0/2093453870 shutdown_connections 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.972+0000 7f3f4da00640 1 -- 172.21.15.43:0/2093453870 >> 172.21.15.43:0/2093453870 conn(0x7f3f48068090 msgr2=0x7f3f480684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.972+0000 7f3f4da00640 1 -- 172.21.15.43:0/2093453870 shutdown_connections 2024-04-26T20:27:22.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.972+0000 7f3f4da00640 1 -- 172.21.15.43:0/2093453870 wait complete. 2024-04-26T20:27:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.973+0000 7f3f4da00640 1 Processor -- start 2024-04-26T20:27:22.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.974+0000 7f3f4da00640 1 -- start start 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c004d40 0x7f3f48157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f481588a0 0x7f3f4814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480b4b90 con 0x7f3f480b20d0 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4815e010 con 0x7f3f2c004d40 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4da00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480b5890 con 0x7f3f481588a0 2024-04-26T20:27:22.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41446/0 (socket says 172.21.15.43:41446) 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 -- 172.21.15.43:0/3452529591 learned_addr learned my addr 172.21.15.43:0/3452529591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f47e00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c004d40 0x7f3f48157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f47400640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f481588a0 0x7f3f4814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 -- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f481588a0 msgr2=0x7f3f4814c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f481588a0 0x7f3f4814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 -- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c004d40 msgr2=0x7f3f48157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c004d40 0x7f3f48157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.975+0000 7f3f4d000640 1 -- 172.21.15.43:0/3452529591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f48158d20 con 0x7f3f480b20d0 2024-04-26T20:27:22.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f4d000640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000aa40 tx=0x7f3f480b4020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000a4c0 con 0x7f3f480b20d0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f40004b60 con 0x7f3f480b20d0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f480ccc70 con 0x7f3f480b20d0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40002000 con 0x7f3f480b20d0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.976+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f48156c60 con 0x7f3f480b20d0 2024-04-26T20:27:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.978+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480506c0 con 0x7f3f480b20d0 2024-04-26T20:27:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.979+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f40034470 con 0x7f3f480b20d0 2024-04-26T20:27:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.980+0000 7f3f3fe00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 0x7f3f2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.980+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f3f400ac1e0 con 0x7f3f480b20d0 2024-04-26T20:27:22.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.980+0000 7f3f47e00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 0x7f3f2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:22.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.981+0000 7f3f47e00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 0x7f3f2406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009b80 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:22.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:22.985+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f4007a1d0 con 0x7f3f480b20d0 2024-04-26T20:27:23.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.188+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f48159f90 con 0x7f3f480b20d0 2024-04-26T20:27:23.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.189+0000 7f3f3fe00640 1 -- 172.21.15.43:0/3452529591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f3f40079ad0 con 0x7f3f480b20d0 2024-04-26T20:27:23.190 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 msgr2=0x7f3f2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 0x7f3f2406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009b80 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-04-26T20:27:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 msgr2=0x7f3f480b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000aa40 tx=0x7f3f480b4020 comp rx=0 tx=0).stop 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 shutdown_connections 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3f2406c8f0 0x7f3f2406edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3f481588a0 0x7f3f4814c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f480b20d0 0x7f3f480b2550 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 --2- 172.21.15.43:0/3452529591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3f2c004d40 0x7f3f48157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 >> 172.21.15.43:0/3452529591 conn(0x7f3f48068090 msgr2=0x7f3f48071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 shutdown_connections 2024-04-26T20:27:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.192+0000 7f3f4da00640 1 -- 172.21.15.43:0/3452529591 wait complete. 2024-04-26T20:27:23.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.424+0000 7f90cb200640 1 Processor -- start 2024-04-26T20:27:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.424+0000 7f90cb200640 1 -- start start 2024-04-26T20:27:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40720c0 0x7f90c40b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90c40b5c90 0x7f90c40ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 0x7f90c40724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f90c40b46b0 con 0x7f90c40b5c90 2024-04-26T20:27:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f90c40b4990 con 0x7f90c40ce840 2024-04-26T20:27:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90cb200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f90c40b4c70 con 0x7f90c40720c0 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.425+0000 7f90c3400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90c40b5c90 0x7f90c40ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90c40b5c90 0x7f90c40ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41458/0 (socket says 172.21.15.43:41458) 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 -- 172.21.15.43:0/2987478784 learned_addr learned my addr 172.21.15.43:0/2987478784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c9200640 1 --2- 172.21.15.43:0/2987478784 >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 0x7f90c40724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3e00640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40720c0 0x7f90c40b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 -- 172.21.15.43:0/2987478784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40720c0 msgr2=0x7f90c40b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40720c0 0x7f90c40b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 -- 172.21.15.43:0/2987478784 >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 msgr2=0x7f90c40724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 --2- 172.21.15.43:0/2987478784 >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 0x7f90c40724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c3400640 1 -- 172.21.15.43:0/2987478784 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c40b4f50 con 0x7f90c40b5c90 2024-04-26T20:27:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.426+0000 7f90c9200640 1 --2- 172.21.15.43:0/2987478784 >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 0x7f90c40724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.427+0000 7f90c3400640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c40b5c90 0x7f90c40ce300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b3760 tx=0x7f90ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee45abb187033c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90c2a00640 1 -- 172.21.15.43:0/2987478784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac037070 con 0x7f90c40b5c90 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90c2a00640 1 -- 172.21.15.43:0/2987478784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac032440 con 0x7f90c40b5c90 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90c2a00640 1 -- 172.21.15.43:0/2987478784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac032810 con 0x7f90c40b5c90 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 -- 172.21.15.43:0/2987478784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c40b5c90 msgr2=0x7f90c40ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c40b5c90 0x7f90c40ce300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90c40b3760 tx=0x7f90ac02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 -- 172.21.15.43:0/2987478784 shutdown_connections 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 --2- 172.21.15.43:0/2987478784 >> v2:172.21.15.164:3300/0 conn(0x7f90c40ce840 0x7f90c40724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c40b5c90 0x7f90c40ce300 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 --2- 172.21.15.43:0/2987478784 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40720c0 0x7f90c40b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 -- 172.21.15.43:0/2987478784 >> 172.21.15.43:0/2987478784 conn(0x7f90c40b2900 msgr2=0x7f90c40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 -- 172.21.15.43:0/2987478784 shutdown_connections 2024-04-26T20:27:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.428+0000 7f90cb200640 1 -- 172.21.15.43:0/2987478784 wait complete. 2024-04-26T20:27:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.430+0000 7f90cb200640 1 Processor -- start 2024-04-26T20:27:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.430+0000 7f90cb200640 1 -- start start 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c40720c0 0x7f90c4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40ce840 0x7f90c41f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40729e0 con 0x7f90c41f2a10 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40cecf0 con 0x7f90c40720c0 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.431+0000 7f90cb200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f90c4072b60 con 0x7f90c40ce840 2024-04-26T20:27:23.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41460/0 (socket says 172.21.15.43:41460) 2024-04-26T20:27:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 -- 172.21.15.43:0/1864724717 learned_addr learned my addr 172.21.15.43:0/1864724717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c3e00640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c40720c0 0x7f90c4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c3400640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40ce840 0x7f90c41f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 -- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40ce840 msgr2=0x7f90c41f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40ce840 0x7f90c41f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 -- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c40720c0 msgr2=0x7f90c4148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c40720c0 0x7f90c4148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.432+0000 7f90c9200640 1 -- 172.21.15.43:0/1864724717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c41f2f90 con 0x7f90c41f2a10 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90c9200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00cef0 tx=0x7f90bc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01a070 con 0x7f90c41f2a10 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc00d860 con 0x7f90c41f2a10 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00dc50 con 0x7f90c41f2a10 2024-04-26T20:27:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c41f3270 con 0x7f90c41f2a10 2024-04-26T20:27:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.433+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41f3830 con 0x7f90c41f2a10 2024-04-26T20:27:23.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.434+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c4069350 con 0x7f90c41f2a10 2024-04-26T20:27:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.435+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90bc01d070 con 0x7f90c41f2a10 2024-04-26T20:27:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.436+0000 7f90bbe00640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 0x7f90a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.436+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f90bc08c8f0 con 0x7f90c41f2a10 2024-04-26T20:27:23.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.436+0000 7f90c3e00640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 0x7f90a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.437+0000 7f90c3e00640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 0x7f90a006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.442+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90bc05a910 con 0x7f90c41f2a10 2024-04-26T20:27:23.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.650+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90c40b43f0 con 0x7f90c41f2a10 2024-04-26T20:27:23.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.651+0000 7f90bbe00640 1 -- 172.21.15.43:0/1864724717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f90bc05a210 con 0x7f90c41f2a10 2024-04-26T20:27:23.652 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 msgr2=0x7f90a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 0x7f90a006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4008040 comp rx=0 tx=0).stop 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 msgr2=0x7f90c414c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00cef0 tx=0x7f90bc00a710 comp rx=0 tx=0).stop 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 shutdown_connections 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f90a006c8f0 0x7f90a006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90c40ce840 0x7f90c41f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90c41f2a10 0x7f90c414c6b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 --2- 172.21.15.43:0/1864724717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90c40720c0 0x7f90c4148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 >> 172.21.15.43:0/1864724717 conn(0x7f90c40b2900 msgr2=0x7f90c4150860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 shutdown_connections 2024-04-26T20:27:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.655+0000 7f90cb200640 1 -- 172.21.15.43:0/1864724717 wait complete. 2024-04-26T20:27:23.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.889+0000 7f3553e00640 1 Processor -- start 2024-04-26T20:27:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.890+0000 7f3553e00640 1 -- start start 2024-04-26T20:27:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.891+0000 7f3553e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f355415a7e0 0x7f35540b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.891+0000 7f3553e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3553e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35540b3900 con 0x7f355415d2b0 2024-04-26T20:27:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3553e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f355415abc0 con 0x7f355415a7e0 2024-04-26T20:27:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3553e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f35540b3b70 con 0x7f35540b5410 2024-04-26T20:27:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3552000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3552000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:53962/0 (socket says 172.21.15.43:53962) 2024-04-26T20:27:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.892+0000 7f3552000640 1 -- 172.21.15.43:0/3627849619 learned_addr learned my addr 172.21.15.43:0/3627849619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.893+0000 7f3552a00640 1 --2- 172.21.15.43:0/3627849619 >> v2:172.21.15.164:3300/0 conn(0x7f355415a7e0 0x7f35540b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.893+0000 7f3552000640 1 -- 172.21.15.43:0/3627849619 >> v2:172.21.15.164:3300/0 conn(0x7f355415a7e0 msgr2=0x7f35540b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.893+0000 7f3552000640 1 --2- 172.21.15.43:0/3627849619 >> v2:172.21.15.164:3300/0 conn(0x7f355415a7e0 0x7f35540b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.893+0000 7f3552000640 1 -- 172.21.15.43:0/3627849619 >> v1:172.21.15.164:6789/0 conn(0x7f355415d2b0 legacy=0x7f355415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.893+0000 7f3552000640 1 -- 172.21.15.43:0/3627849619 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540b3e50 con 0x7f35540b5410 2024-04-26T20:27:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.894+0000 7f3552000640 1 --2- 172.21.15.43:0/3627849619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3540009870 tx=0x7f354002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b8eab60d8f287bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.895+0000 7f3551600640 1 -- 172.21.15.43:0/3627849619 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540037070 con 0x7f35540b5410 2024-04-26T20:27:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.895+0000 7f3551600640 1 -- 172.21.15.43:0/3627849619 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3540032440 con 0x7f35540b5410 2024-04-26T20:27:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.895+0000 7f3551600640 1 -- 172.21.15.43:0/3627849619 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540032810 con 0x7f35540b5410 2024-04-26T20:27:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 -- 172.21.15.43:0/3627849619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 msgr2=0x7f35540b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 --2- 172.21.15.43:0/3627849619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3540009870 tx=0x7f354002a250 comp rx=0 tx=0).stop 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 -- 172.21.15.43:0/3627849619 shutdown_connections 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 --2- 172.21.15.43:0/3627849619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f35540b5410 0x7f35540b5890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 --2- 172.21.15.43:0/3627849619 >> v2:172.21.15.164:3300/0 conn(0x7f355415a7e0 0x7f35540b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 -- 172.21.15.43:0/3627849619 >> 172.21.15.43:0/3627849619 conn(0x7f3554068090 msgr2=0x7f35540684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 -- 172.21.15.43:0/3627849619 shutdown_connections 2024-04-26T20:27:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.896+0000 7f3553e00640 1 -- 172.21.15.43:0/3627849619 wait complete. 2024-04-26T20:27:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.899+0000 7f3553e00640 1 Processor -- start 2024-04-26T20:27:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.899+0000 7f3553e00640 1 -- start start 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.900+0000 7f3553e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.900+0000 7f3553e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 0x7f35541f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 0x7f35540b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f35540b4b90 con 0x7f35540b5410 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f355415e010 con 0x7f355415a7e0 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f355415e2e0 con 0x7f355415d2b0 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41466/0 (socket says 172.21.15.43:41466) 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 -- 172.21.15.43:0/3290052309 learned_addr learned my addr 172.21.15.43:0/3290052309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552000640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 0x7f35541f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553400640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 0x7f35540b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 -- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 msgr2=0x7f35540b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 0x7f35540b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 -- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 msgr2=0x7f35541f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 0x7f35541f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552a00640 1 -- 172.21.15.43:0/3290052309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35541f99c0 con 0x7f35540b5410 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3552000640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 0x7f35541f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.901+0000 7f3553400640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 0x7f35540b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f3552a00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9b0 tx=0x7f354400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544017070 con 0x7f35540b5410 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354400c9a0 con 0x7f35540b5410 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354400cd90 con 0x7f35540b5410 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35541f9ca0 con 0x7f35540b5410 2024-04-26T20:27:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.902+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541fa540 con 0x7f35540b5410 2024-04-26T20:27:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.903+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35541f9e70 con 0x7f35540b5410 2024-04-26T20:27:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.904+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f354401a070 con 0x7f35540b5410 2024-04-26T20:27:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.905+0000 7f354aa00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.905+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f354408c3f0 con 0x7f35540b5410 2024-04-26T20:27:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.906+0000 7f3552000640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.906+0000 7f3552000640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3540034000 tx=0x7f354002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:23.910+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f354405a350 con 0x7f35540b5410 2024-04-26T20:27:24.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.118+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f35540506c0 con 0x7f35540b5410 2024-04-26T20:27:24.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.119+0000 7f354aa00640 1 -- 172.21.15.43:0/3290052309 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f3544059c50 con 0x7f35540b5410 2024-04-26T20:27:24.120 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 msgr2=0x7f353406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 0x7f353406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3540034000 tx=0x7f354002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 msgr2=0x7f355414a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f354400e9b0 tx=0x7f354400ee70 comp rx=0 tx=0).stop 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 shutdown_connections 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f353406c8f0 0x7f353406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f355415d2b0 0x7f35540b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f355415a7e0 0x7f35541f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 --2- 172.21.15.43:0/3290052309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35540b5410 0x7f355414a8d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 >> 172.21.15.43:0/3290052309 conn(0x7f3554068090 msgr2=0x7f35540afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 shutdown_connections 2024-04-26T20:27:24.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.122+0000 7f3553e00640 1 -- 172.21.15.43:0/3290052309 wait complete. 2024-04-26T20:27:24.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.358+0000 7f6eda200640 1 Processor -- start 2024-04-26T20:27:24.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.358+0000 7f6eda200640 1 -- start start 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed40cd180 0x7f6ed4072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ed415dc50 0x7f6ed40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6ed40ce2b0 0x7f6ed41607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed40cd560 con 0x7f6ed415dc50 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed4072d00 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6eda200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4072fe0 con 0x7f6ed40cd180 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6ed3e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ed415dc50 0x7f6ed40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6ed3e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ed415dc50 0x7f6ed40cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41472/0 (socket says 172.21.15.43:41472) 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.360+0000 7f6ed3e00640 1 -- 172.21.15.43:0/922155007 learned_addr learned my addr 172.21.15.43:0/922155007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed9800640 1 --2- 172.21.15.43:0/922155007 >> v2:172.21.15.164:3300/0 conn(0x7f6ed40ce2b0 0x7f6ed41607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed8e00640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed40cd180 0x7f6ed4072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed3e00640 1 -- 172.21.15.43:0/922155007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed40cd180 msgr2=0x7f6ed4072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed40cd180 0x7f6ed4072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed3e00640 1 -- 172.21.15.43:0/922155007 >> v2:172.21.15.164:3300/0 conn(0x7f6ed40ce2b0 msgr2=0x7f6ed41607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:24.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/922155007 >> v2:172.21.15.164:3300/0 conn(0x7f6ed40ce2b0 0x7f6ed41607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.361+0000 7f6ed3e00640 1 -- 172.21.15.43:0/922155007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40732c0 con 0x7f6ed415dc50 2024-04-26T20:27:24.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.362+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed415dc50 0x7f6ed40cdd70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4072080 tx=0x7f6ec402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823a4b43795f9d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.362+0000 7f6ed3400640 1 -- 172.21.15.43:0/922155007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037070 con 0x7f6ed415dc50 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.362+0000 7f6ed3400640 1 -- 172.21.15.43:0/922155007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4032440 con 0x7f6ed415dc50 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.362+0000 7f6ed3400640 1 -- 172.21.15.43:0/922155007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4032810 con 0x7f6ed415dc50 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 -- 172.21.15.43:0/922155007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed415dc50 msgr2=0x7f6ed40cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed415dc50 0x7f6ed40cdd70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4072080 tx=0x7f6ec402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 -- 172.21.15.43:0/922155007 shutdown_connections 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 --2- 172.21.15.43:0/922155007 >> v2:172.21.15.164:3300/0 conn(0x7f6ed40ce2b0 0x7f6ed41607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed415dc50 0x7f6ed40cdd70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 --2- 172.21.15.43:0/922155007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed40cd180 0x7f6ed4072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 -- 172.21.15.43:0/922155007 >> 172.21.15.43:0/922155007 conn(0x7f6ed4068090 msgr2=0x7f6ed4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 -- 172.21.15.43:0/922155007 shutdown_connections 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.363+0000 7f6eda200640 1 -- 172.21.15.43:0/922155007 wait complete. 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.365+0000 7f6eda200640 1 Processor -- start 2024-04-26T20:27:24.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.365+0000 7f6eda200640 1 -- start start 2024-04-26T20:27:24.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ed40cd180 0x7f6ed415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 0x7f6ed413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed415e0d0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4073da0 con 0x7f6ed40cd180 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6eda200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4160d10 con 0x7f6ed415dc50 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6ed3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41488/0 (socket says 172.21.15.43:41488) 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.366+0000 7f6ed3e00640 1 -- 172.21.15.43:0/3483100159 learned_addr learned my addr 172.21.15.43:0/3483100159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed8e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ed40cd180 0x7f6ed415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed9800640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 0x7f6ed413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 -- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 msgr2=0x7f6ed413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 0x7f6ed413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 -- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ed40cd180 msgr2=0x7f6ed415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ed40cd180 0x7f6ed415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 -- 172.21.15.43:0/3483100159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed413ed40 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed9800640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 0x7f6ed413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.367+0000 7f6ed3e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402a220 tx=0x7f6ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.368+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4041020 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.368+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec40306c0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.368+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037040 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.368+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed413f020 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.368+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed413f5e0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.369+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed4069350 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.370+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ec4044020 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.371+0000 7f6ed0c00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:24.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.371+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f6ec40ac7e0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.371+0000 7f6ed8e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:24.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.372+0000 7f6ed8e00640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006fd0 tx=0x7f6ec8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:24.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.375+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ec404b030 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.597+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6ed40720e0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.597+0000 7f6ed0c00640 1 -- 172.21.15.43:0/3483100159 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f6ec407a7d0 con 0x7f6ed40ce2b0 2024-04-26T20:27:24.598 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 msgr2=0x7f6eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006fd0 tx=0x7f6ec8008040 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 msgr2=0x7f6ed413ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402a220 tx=0x7f6ec40046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 shutdown_connections 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ed415dc50 0x7f6ed413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed40ce2b0 0x7f6ed413ffd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 --2- 172.21.15.43:0/3483100159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ed40cd180 0x7f6ed415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 >> 172.21.15.43:0/3483100159 conn(0x7f6ed4068090 msgr2=0x7f6ed40cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.600+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 shutdown_connections 2024-04-26T20:27:24.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:24.601+0000 7f6eda200640 1 -- 172.21.15.43:0/3483100159 wait complete. 2024-04-26T20:27:24.622 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-26T20:27:24.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:24.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-26T20:27:24.708 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:24.708 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:27:24.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:24.708 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T20:27:24.764 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:24.764 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:24.822 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:24.822 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:24.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.101+0000 7f9d0e800640 1 Processor -- start 2024-04-26T20:27:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.101+0000 7f9d0e800640 1 -- start start 2024-04-26T20:27:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0e800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9d080cd180 0x7f9d08072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0e800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d080cd560 con 0x7f9d080ce480 2024-04-26T20:27:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0e800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d08072d00 con 0x7f9d0815dc50 2024-04-26T20:27:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0e800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9d08072fe0 con 0x7f9d080cd180 2024-04-26T20:27:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0de00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d080ce480 0x7f9d0806dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58242/0 (socket says 172.21.15.43:58242) 2024-04-26T20:27:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.103+0000 7f9d0de00640 1 -- 172.21.15.43:0/3952558124 learned_addr learned my addr 172.21.15.43:0/3952558124 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3952558124 >> v2:172.21.15.164:3300/0 conn(0x7f9d080cd180 0x7f9d08072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685756370 0 0) 0x7f9d080cd560 con 0x7f9d080ce480 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cf0003520 con 0x7f9d080ce480 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3434259750 0 0) 0x7f9cf0003520 con 0x7f9d080ce480 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 >> v2:172.21.15.164:3300/0 conn(0x7f9d080cd180 msgr2=0x7f9d08072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 --2- 172.21.15.43:0/3952558124 >> v2:172.21.15.164:3300/0 conn(0x7f9d080cd180 0x7f9d08072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 >> v1:172.21.15.164:6789/0 conn(0x7f9d0815dc50 legacy=0x7f9d080cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d080732c0 con 0x7f9d080ce480 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f9d000019f0 con 0x7f9d080ce480 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 >> v1:172.21.15.43:6789/0 conn(0x7f9d080ce480 legacy=0x7f9d0806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf00039b0 0x7f9cf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.104+0000 7f9d07e00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d07e00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf00051c0 0x7f9cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08072fe0 con 0x7f9cf0004800 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf000c060 con 0x7f9cf00039b0 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf000c260 con 0x7f9cf00051c0 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 -- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf00051c0 msgr2=0x7f9cf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf00051c0 0x7f9cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 -- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf00039b0 msgr2=0x7f9cf0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf00039b0 0x7f9cf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9cf00039b0 0x7f9cf0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0ca00640 1 -- 172.21.15.43:0/3952558124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf0003520 con 0x7f9cf0004800 2024-04-26T20:27:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.105+0000 7f9d0de00640 1 -- 172.21.15.43:0/3952558124 reap_dead start 2024-04-26T20:27:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.106+0000 7f9d0ca00640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4001750 tx=0x7f9cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c67706dc47c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4001370 con 0x7f9cf0004800 2024-04-26T20:27:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf40052d0 con 0x7f9cf0004800 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d07e00640 1 -- 172.21.15.43:0/3952558124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf402abe0 con 0x7f9cf0004800 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d0e800640 1 -- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 msgr2=0x7f9cf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4001750 tx=0x7f9cf0005830 comp rx=0 tx=0).stop 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d0e800640 1 -- 172.21.15.43:0/3952558124 shutdown_connections 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3952558124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.107+0000 7f9d0e800640 1 -- 172.21.15.43:0/3952558124 >> 172.21.15.43:0/3952558124 conn(0x7f9d08068090 msgr2=0x7f9d08068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.108+0000 7f9d0e800640 1 -- 172.21.15.43:0/3952558124 shutdown_connections 2024-04-26T20:27:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.108+0000 7f9d0e800640 1 -- 172.21.15.43:0/3952558124 wait complete. 2024-04-26T20:27:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.109+0000 7f9d0e800640 1 Processor -- start 2024-04-26T20:27:25.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.110+0000 7f9d0e800640 1 -- start start 2024-04-26T20:27:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 0x7f9d080cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d080b4660 0x7f9d080b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0815e0d0 con 0x7f9d080c5a40 2024-04-26T20:27:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08073da0 con 0x7f9d080b4660 2024-04-26T20:27:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0e800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9d08072620 con 0x7f9cf0004800 2024-04-26T20:27:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41504/0 (socket says 172.21.15.43:41504) 2024-04-26T20:27:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 -- 172.21.15.43:0/3248713538 learned_addr learned my addr 172.21.15.43:0/3248713538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 0x7f9d080cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0ca00640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d080b4660 0x7f9d080b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 -- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 msgr2=0x7f9d080cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 0x7f9d080cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 -- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d080b4660 msgr2=0x7f9d080b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d080b4660 0x7f9d080b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0de00640 1 -- 172.21.15.43:0/3248713538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d080b56c0 con 0x7f9d080c5a40 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.111+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 0x7f9d080cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d0de00640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b5c50 tx=0x7f9d0000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004d60 con 0x7f9d080c5a40 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0000a610 con 0x7f9d080c5a40 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d080b5100 con 0x7f9d080c5a40 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000aa00 con 0x7f9d080c5a40 2024-04-26T20:27:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.112+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d08134d30 con 0x7f9d080c5a40 2024-04-26T20:27:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.114+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d08069350 con 0x7f9d080c5a40 2024-04-26T20:27:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.115+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d00011070 con 0x7f9d080c5a40 2024-04-26T20:27:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.116+0000 7f9d05600640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.116+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f9d0008ba50 con 0x7f9d080c5a40 2024-04-26T20:27:25.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.116+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.117+0000 7f9d0d400640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 0x7f9ce806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d08051930 tx=0x7f9cfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.120+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d00059ae0 con 0x7f9d080c5a40 2024-04-26T20:27:25.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.328+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d080d0670 con 0x7f9d080c5a40 2024-04-26T20:27:25.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.330+0000 7f9d05600640 1 -- 172.21.15.43:0/3248713538 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f9d000593e0 con 0x7f9d080c5a40 2024-04-26T20:27:25.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:25.331 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 msgr2=0x7f9ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 0x7f9ce806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9d08051930 tx=0x7f9cfc008040 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 msgr2=0x7f9d080c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9d080b5c50 tx=0x7f9d0000cbe0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 shutdown_connections 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9ce806c8f0 0x7f9ce806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d080c5a40 0x7f9d080c5ec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d080b4660 0x7f9d080b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 --2- 172.21.15.43:0/3248713538 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9cf0004800 0x7f9d080cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 >> 172.21.15.43:0/3248713538 conn(0x7f9d08068090 msgr2=0x7f9d080b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 shutdown_connections 2024-04-26T20:27:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.333+0000 7f9d0e800640 1 -- 172.21.15.43:0/3248713538 wait complete. 2024-04-26T20:27:25.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:25.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:25.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.568+0000 7f2f59a00640 1 Processor -- start 2024-04-26T20:27:25.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.568+0000 7f2f59a00640 1 -- start start 2024-04-26T20:27:25.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.569+0000 7f2f59a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 0x7f2f540b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.569+0000 7f2f59a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f5415d2b0 0x7f2f5415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.569+0000 7f2f59a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f540b3900 con 0x7f2f5415a7e0 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.569+0000 7f2f59a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5415abc0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.569+0000 7f2f59a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2f540b3b70 con 0x7f2f540b5410 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f5415d2b0 0x7f2f5415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f5415d2b0 0x7f2f5415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41506/0 (socket says 172.21.15.43:41506) 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 -- 172.21.15.43:0/2154036829 learned_addr learned my addr 172.21.15.43:0/2154036829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f53400640 1 --2- 172.21.15.43:0/2154036829 >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 0x7f2f540b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 -- 172.21.15.43:0/2154036829 >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 msgr2=0x7f2f540b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 --2- 172.21.15.43:0/2154036829 >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 0x7f2f540b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 -- 172.21.15.43:0/2154036829 >> v1:172.21.15.164:6789/0 conn(0x7f2f5415a7e0 legacy=0x7f2f540b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f59000640 1 -- 172.21.15.43:0/2154036829 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f540b3e50 con 0x7f2f5415d2b0 2024-04-26T20:27:25.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.570+0000 7f2f53400640 1 --2- 172.21.15.43:0/2154036829 >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 0x7f2f540b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.571+0000 7f2f59000640 1 --2- 172.21.15.43:0/2154036829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f5415d740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2f540d15f0 tx=0x7f2f4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46744ba8e0566712 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f52a00640 1 -- 172.21.15.43:0/2154036829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c036070 con 0x7f2f5415d2b0 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f52a00640 1 -- 172.21.15.43:0/2154036829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c031600 con 0x7f2f5415d2b0 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f52a00640 1 -- 172.21.15.43:0/2154036829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0319d0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 -- 172.21.15.43:0/2154036829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 msgr2=0x7f2f5415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 --2- 172.21.15.43:0/2154036829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f5415d740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2f540d15f0 tx=0x7f2f4c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 -- 172.21.15.43:0/2154036829 shutdown_connections 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 --2- 172.21.15.43:0/2154036829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f5415d740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 --2- 172.21.15.43:0/2154036829 >> v2:172.21.15.164:3300/0 conn(0x7f2f540b5410 0x7f2f540b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.572+0000 7f2f59a00640 1 -- 172.21.15.43:0/2154036829 >> 172.21.15.43:0/2154036829 conn(0x7f2f54068090 msgr2=0x7f2f540684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.573+0000 7f2f59a00640 1 -- 172.21.15.43:0/2154036829 shutdown_connections 2024-04-26T20:27:25.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.573+0000 7f2f59a00640 1 -- 172.21.15.43:0/2154036829 wait complete. 2024-04-26T20:27:25.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.574+0000 7f2f59a00640 1 Processor -- start 2024-04-26T20:27:25.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.574+0000 7f2f59a00640 1 -- start start 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.575+0000 7f2f59a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f540b5410 0x7f2f541f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.575+0000 7f2f59a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 0x7f2f541f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.575+0000 7f2f59a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5415dc80 con 0x7f2f5415d2b0 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5415e010 con 0x7f2f5415a7e0 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5415e2e0 con 0x7f2f540b5410 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41514/0 (socket says 172.21.15.43:41514) 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 -- 172.21.15.43:0/1471674952 learned_addr learned my addr 172.21.15.43:0/1471674952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f53400640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 0x7f2f541f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 -- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f540b5410 msgr2=0x7f2f541f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f540b5410 0x7f2f541f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 -- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 msgr2=0x7f2f541f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 0x7f2f541f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f59000640 1 -- 172.21.15.43:0/1471674952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f541f82c0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.576+0000 7f2f53400640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 0x7f2f541f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f59000640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c02ace0 tx=0x7f2f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c036050 con 0x7f2f5415d2b0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c042070 con 0x7f2f5415d2b0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c009510 con 0x7f2f5415d2b0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f541f8540 con 0x7f2f5415d2b0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.577+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541f8b00 con 0x7f2f5415d2b0 2024-04-26T20:27:25.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.579+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f540b5870 con 0x7f2f5415d2b0 2024-04-26T20:27:25.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.579+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f4c007bd0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.581+0000 7f2f4be00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 0x7f2f3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:25.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.581+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f2f4c0acf80 con 0x7f2f5415d2b0 2024-04-26T20:27:25.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.581+0000 7f2f53e00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 0x7f2f3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:25.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.582+0000 7f2f53e00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 0x7f2f3006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f54073170 tx=0x7f2f44007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.586+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f540b5870 con 0x7f2f5415d2b0 2024-04-26T20:27:25.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.790+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f540506c0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.791+0000 7f2f4be00640 1 -- 172.21.15.43:0/1471674952 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2f541f82c0 con 0x7f2f5415d2b0 2024-04-26T20:27:25.792 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 msgr2=0x7f2f3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 0x7f2f3006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f54073170 tx=0x7f2f44007990 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 msgr2=0x7f2f540aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c02ace0 tx=0x7f2f4c004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 shutdown_connections 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2f3006c8f0 0x7f2f3006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f5415d2b0 0x7f2f540aeb30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f5415a7e0 0x7f2f541f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 --2- 172.21.15.43:0/1471674952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f540b5410 0x7f2f541f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 >> 172.21.15.43:0/1471674952 conn(0x7f2f54068090 msgr2=0x7f2f5415f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 shutdown_connections 2024-04-26T20:27:25.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:25.794+0000 7f2f59a00640 1 -- 172.21.15.43:0/1471674952 wait complete. 2024-04-26T20:27:25.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:26.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.040+0000 7f4805e00640 1 Processor -- start 2024-04-26T20:27:26.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.040+0000 7f4805e00640 1 -- start start 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f4805e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48000b5bb0 0x7f48000d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f4805e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48000b6100 0x7f48000cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f4805e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48000b6800 con 0x7f48000d2f70 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f4805e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f48000d5ea0 con 0x7f48000b6100 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f4805e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f48000d3f20 con 0x7f48000b5bb0 2024-04-26T20:27:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f47fea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f48000d2f70 0x7f48000d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36636/0 (socket says 172.21.15.43:36636) 2024-04-26T20:27:26.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.041+0000 7f47fea00640 1 -- 172.21.15.43:0/2313268264 learned_addr learned my addr 172.21.15.43:0/2313268264 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:26.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47ff400640 1 --2- 172.21.15.43:0/2313268264 >> v2:172.21.15.43:3300/0 conn(0x7f48000b6100 0x7f48000cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47fe000640 1 --2- 172.21.15.43:0/2313268264 >> v2:172.21.15.164:3300/0 conn(0x7f48000b5bb0 0x7f48000d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47fd600640 1 -- 172.21.15.43:0/2313268264 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431116876 0 0) 0x7f47f0001110 con 0x7f48000d2f70 2024-04-26T20:27:26.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47fd600640 1 -- 172.21.15.43:0/2313268264 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47e0003520 con 0x7f48000d2f70 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47ff400640 1 -- 172.21.15.43:0/2313268264 >> v2:172.21.15.164:3300/0 conn(0x7f48000b5bb0 msgr2=0x7f48000d3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47ff400640 1 --2- 172.21.15.43:0/2313268264 >> v2:172.21.15.164:3300/0 conn(0x7f48000b5bb0 0x7f48000d3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47ff400640 1 -- 172.21.15.43:0/2313268264 >> v1:172.21.15.43:6789/0 conn(0x7f48000d2f70 legacy=0x7f48000d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.042+0000 7f47ff400640 1 -- 172.21.15.43:0/2313268264 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48000d0ff0 con 0x7f48000b6100 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.043+0000 7f47ff400640 1 --2- 172.21.15.43:0/2313268264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b6100 0x7f48000cbff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f480008d2a0 tx=0x7f47f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cfd6a4b4ecee45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.043+0000 7f47fd600640 1 -- 172.21.15.43:0/2313268264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8037070 con 0x7f48000b6100 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f47fd600640 1 -- 172.21.15.43:0/2313268264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f8032440 con 0x7f48000b6100 2024-04-26T20:27:26.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f47fd600640 1 -- 172.21.15.43:0/2313268264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8032810 con 0x7f48000b6100 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 -- 172.21.15.43:0/2313268264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b6100 msgr2=0x7f48000cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 --2- 172.21.15.43:0/2313268264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b6100 0x7f48000cbff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f480008d2a0 tx=0x7f47f802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 -- 172.21.15.43:0/2313268264 shutdown_connections 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 --2- 172.21.15.43:0/2313268264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b6100 0x7f48000cbff0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 --2- 172.21.15.43:0/2313268264 >> v2:172.21.15.164:3300/0 conn(0x7f48000b5bb0 0x7f48000d3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 -- 172.21.15.43:0/2313268264 >> 172.21.15.43:0/2313268264 conn(0x7f48000d4f20 msgr2=0x7f48000d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 -- 172.21.15.43:0/2313268264 shutdown_connections 2024-04-26T20:27:26.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.044+0000 7f4805e00640 1 -- 172.21.15.43:0/2313268264 wait complete. 2024-04-26T20:27:26.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.046+0000 7f4805e00640 1 Processor -- start 2024-04-26T20:27:26.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.046+0000 7f4805e00640 1 -- start start 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48000b6100 0x7f48000ba480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48000d2f70 0x7f48000ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48000d3c40 con 0x7f48000b5bb0 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48000d3960 con 0x7f48000b6100 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.047+0000 7f4805e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f48000cc530 con 0x7f48000d2f70 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60960/0 (socket says 172.21.15.43:60960) 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 -- 172.21.15.43:0/1881198512 learned_addr learned my addr 172.21.15.43:0/1881198512 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fe000640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48000b6100 0x7f48000ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47ff400640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48000d2f70 0x7f48000ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 -- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48000d2f70 msgr2=0x7f48000ba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48000d2f70 0x7f48000ba9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 -- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48000b6100 msgr2=0x7f48000ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48000b6100 0x7f48000ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.048+0000 7f47fea00640 1 -- 172.21.15.43:0/1881198512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48001e4a70 con 0x7f48000b5bb0 2024-04-26T20:27:26.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f47fea00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c3a0 tx=0x7f47f000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0017070 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0005ce0 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48000ca420 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0007310 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.049+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48000ca9e0 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.050+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800069a60 con 0x7f48000b5bb0 2024-04-26T20:27:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.051+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47f001a070 con 0x7f48000b5bb0 2024-04-26T20:27:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.052+0000 7f47f7400640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 0x7f47dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.052+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f47f008b460 con 0x7f48000b5bb0 2024-04-26T20:27:26.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.053+0000 7f47fe000640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 0x7f47dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.053+0000 7f47fe000640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 0x7f47dc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48000d25c0 tx=0x7f47e8003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.057+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48000b6800 con 0x7f48000b5bb0 2024-04-26T20:27:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.261+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48000d1fc0 con 0x7f48000b5bb0 2024-04-26T20:27:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.263+0000 7f47f7400640 1 -- 172.21.15.43:0/1881198512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f47f00593c0 con 0x7f48000b5bb0 2024-04-26T20:27:26.263 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 msgr2=0x7f47dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 0x7f47dc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48000d25c0 tx=0x7f47e8003990 comp rx=0 tx=0).stop 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 msgr2=0x7f4800157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c3a0 tx=0x7f47f000c860 comp rx=0 tx=0).stop 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 shutdown_connections 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f47dc06c8f0 0x7f47dc06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48000d2f70 0x7f48000ba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48000b6100 0x7f48000ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 --2- 172.21.15.43:0/1881198512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48000b5bb0 0x7f4800157fa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.265+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 >> 172.21.15.43:0/1881198512 conn(0x7f48000d4f20 msgr2=0x7f48000ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.266+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 shutdown_connections 2024-04-26T20:27:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.266+0000 7f4805e00640 1 -- 172.21.15.43:0/1881198512 wait complete. 2024-04-26T20:27:26.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.497+0000 7fd656800640 1 Processor -- start 2024-04-26T20:27:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.497+0000 7fd656800640 1 -- start start 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.498+0000 7fd656800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6500b4310 0x7fd650073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.498+0000 7fd656800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500b48f0 con 0x7fd650073740 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd656800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500b4bd0 con 0x7fd650159870 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd656800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd650159c50 con 0x7fd6500b4310 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6500b4310 0x7fd650073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64ea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd650073740 0x7fd6500b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36652/0 (socket says 172.21.15.43:36652) 2024-04-26T20:27:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64ea00640 1 -- 172.21.15.43:0/1925189901 learned_addr learned my addr 172.21.15.43:0/1925189901 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64fe00640 1 -- 172.21.15.43:0/1925189901 >> v1:172.21.15.164:6789/0 conn(0x7fd650159870 legacy=0x7fd65006efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64fe00640 1 -- 172.21.15.43:0/1925189901 >> v1:172.21.15.43:6789/0 conn(0x7fd650073740 legacy=0x7fd6500b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.499+0000 7fd64fe00640 1 -- 172.21.15.43:0/1925189901 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65006e4c0 con 0x7fd6500b4310 2024-04-26T20:27:26.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd64fe00640 1 --2- 172.21.15.43:0/1925189901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6500b4310 0x7fd650073bd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd650159dd0 tx=0x7fd64802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff64e06672ce50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd64e000640 1 -- 172.21.15.43:0/1925189901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648037070 con 0x7fd6500b4310 2024-04-26T20:27:26.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd64e000640 1 -- 172.21.15.43:0/1925189901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd648032440 con 0x7fd6500b4310 2024-04-26T20:27:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd64e000640 1 -- 172.21.15.43:0/1925189901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648032810 con 0x7fd6500b4310 2024-04-26T20:27:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd656800640 1 -- 172.21.15.43:0/1925189901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6500b4310 msgr2=0x7fd650073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.501+0000 7fd656800640 1 --2- 172.21.15.43:0/1925189901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6500b4310 0x7fd650073bd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd650159dd0 tx=0x7fd64802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.502+0000 7fd656800640 1 -- 172.21.15.43:0/1925189901 shutdown_connections 2024-04-26T20:27:26.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.502+0000 7fd656800640 1 --2- 172.21.15.43:0/1925189901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6500b4310 0x7fd650073bd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.502+0000 7fd656800640 1 -- 172.21.15.43:0/1925189901 >> 172.21.15.43:0/1925189901 conn(0x7fd6500b2900 msgr2=0x7fd6500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.502+0000 7fd656800640 1 -- 172.21.15.43:0/1925189901 shutdown_connections 2024-04-26T20:27:26.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.502+0000 7fd656800640 1 -- 172.21.15.43:0/1925189901 wait complete. 2024-04-26T20:27:26.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.503+0000 7fd656800640 1 Processor -- start 2024-04-26T20:27:26.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.504+0000 7fd656800640 1 -- start start 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.504+0000 7fd656800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.504+0000 7fd656800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 0x7fd650147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd656800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 0x7fd650147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd656800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6500720e0 con 0x7fd650073740 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd656800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd650071f60 con 0x7fd650159870 2024-04-26T20:27:26.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd656800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd650072260 con 0x7fd6500b4310 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60988/0 (socket says 172.21.15.43:60988) 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64f400640 1 -- 172.21.15.43:0/2501631964 learned_addr learned my addr 172.21.15.43:0/2501631964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64fe00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 0x7fd650147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64f400640 1 -- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 msgr2=0x7fd650147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64ea00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 0x7fd650147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.505+0000 7fd64f400640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 0x7fd650147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64f400640 1 -- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 msgr2=0x7fd650147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64f400640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 0x7fd650147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64f400640 1 -- 172.21.15.43:0/2501631964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65015c5e0 con 0x7fd650073740 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64fe00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 0x7fd650147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64ea00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 0x7fd650147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd64f400640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd64000ba50 tx=0x7fd64000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640019070 con 0x7fd650073740 2024-04-26T20:27:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd64000cc20 con 0x7fd650073740 2024-04-26T20:27:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640014680 con 0x7fd650073740 2024-04-26T20:27:26.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65015c8c0 con 0x7fd650073740 2024-04-26T20:27:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.506+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65015ce80 con 0x7fd650073740 2024-04-26T20:27:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.508+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd650069350 con 0x7fd650073740 2024-04-26T20:27:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.509+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd64001c070 con 0x7fd650073740 2024-04-26T20:27:26.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.509+0000 7fd647400640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 0x7fd63006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.510+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd64008bcf0 con 0x7fd650073740 2024-04-26T20:27:26.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.510+0000 7fd64ea00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 0x7fd63006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.511+0000 7fd64ea00640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 0x7fd63006ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd650154f10 tx=0x7fd63c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.514+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd640059600 con 0x7fd650073740 2024-04-26T20:27:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.721+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6500b3640 con 0x7fd650073740 2024-04-26T20:27:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.722+0000 7fd647400640 1 -- 172.21.15.43:0/2501631964 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd64005d900 con 0x7fd650073740 2024-04-26T20:27:26.722 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:26.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 msgr2=0x7fd63006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 0x7fd63006ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd650154f10 tx=0x7fd63c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 msgr2=0x7fd65015bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd64000ba50 tx=0x7fd64000bf10 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 shutdown_connections 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd63006c920 0x7fd63006ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd6500b4310 0x7fd650147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd650159870 0x7fd650147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 --2- 172.21.15.43:0/2501631964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd650073740 0x7fd65015bfc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 >> 172.21.15.43:0/2501631964 conn(0x7fd6500b2900 msgr2=0x7fd650073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 shutdown_connections 2024-04-26T20:27:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.725+0000 7fd656800640 1 -- 172.21.15.43:0/2501631964 wait complete. 2024-04-26T20:27:26.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.968+0000 7fc5dbe00640 1 Processor -- start 2024-04-26T20:27:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.968+0000 7fc5dbe00640 1 -- start start 2024-04-26T20:27:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.970+0000 7fc5dbe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 0x7fc5dc06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.970+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.970+0000 7fc5dbe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc15c2a0 con 0x7fc5dc0b4610 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.970+0000 7fc5dbe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc15d060 con 0x7fc5dc15b980 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.970+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc15c580 con 0x7fc5dc15bd60 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.971+0000 7fc5da000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.971+0000 7fc5daa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 0x7fc5dc06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.971+0000 7fc5da000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:48128/0 (socket says 172.21.15.43:48128) 2024-04-26T20:27:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.971+0000 7fc5da000640 1 -- 172.21.15.43:0/529406777 learned_addr learned my addr 172.21.15.43:0/529406777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.972+0000 7fc5da000640 1 -- 172.21.15.43:0/529406777 >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 msgr2=0x7fc5dc06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.972+0000 7fc5da000640 1 --2- 172.21.15.43:0/529406777 >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 0x7fc5dc06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.972+0000 7fc5da000640 1 -- 172.21.15.43:0/529406777 >> v1:172.21.15.164:6789/0 conn(0x7fc5dc0b4610 legacy=0x7fc5dc0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.972+0000 7fc5da000640 1 -- 172.21.15.43:0/529406777 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc15c860 con 0x7fc5dc15bd60 2024-04-26T20:27:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.972+0000 7fc5daa00640 1 --2- 172.21.15.43:0/529406777 >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 0x7fc5dc06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.973+0000 7fc5da000640 1 --2- 172.21.15.43:0/529406777 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009870 tx=0x7fc5c802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9737bab43aa8bc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.974+0000 7fc5d9600640 1 -- 172.21.15.43:0/529406777 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8037070 con 0x7fc5dc15bd60 2024-04-26T20:27:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.974+0000 7fc5d9600640 1 -- 172.21.15.43:0/529406777 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8032440 con 0x7fc5dc15bd60 2024-04-26T20:27:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.974+0000 7fc5d9600640 1 -- 172.21.15.43:0/529406777 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8032810 con 0x7fc5dc15bd60 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.974+0000 7fc5dbe00640 1 -- 172.21.15.43:0/529406777 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 msgr2=0x7fc5dc06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/529406777 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009870 tx=0x7fc5c802a250 comp rx=0 tx=0).stop 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 -- 172.21.15.43:0/529406777 shutdown_connections 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/529406777 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc15bd60 0x7fc5dc06f1d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/529406777 >> v2:172.21.15.164:3300/0 conn(0x7fc5dc15b980 0x7fc5dc06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 -- 172.21.15.43:0/529406777 >> 172.21.15.43:0/529406777 conn(0x7fc5dc068090 msgr2=0x7fc5dc0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 -- 172.21.15.43:0/529406777 shutdown_connections 2024-04-26T20:27:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.975+0000 7fc5dbe00640 1 -- 172.21.15.43:0/529406777 wait complete. 2024-04-26T20:27:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.977+0000 7fc5dbe00640 1 Processor -- start 2024-04-26T20:27:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.977+0000 7fc5dbe00640 1 -- start start 2024-04-26T20:27:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.978+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc0b4610 0x7fc5dc1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.978+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 0x7fc5dc0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc15f6b0 con 0x7fc5dc15b980 2024-04-26T20:27:26.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0b52b0 con 0x7fc5dc15bd60 2024-04-26T20:27:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc15f830 con 0x7fc5dc0b4610 2024-04-26T20:27:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60990/0 (socket says 172.21.15.43:60990) 2024-04-26T20:27:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 -- 172.21.15.43:0/1040348277 learned_addr learned my addr 172.21.15.43:0/1040348277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5db400640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 0x7fc5dc0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5daa00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc0b4610 0x7fc5dc1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 -- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc0b4610 msgr2=0x7fc5dc1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc0b4610 0x7fc5dc1527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 -- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 msgr2=0x7fc5dc0d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 0x7fc5dc0d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5da000640 1 -- 172.21.15.43:0/1040348277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0b24f0 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.979+0000 7fc5db400640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 0x7fc5dc0d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.980+0000 7fc5da000640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c00 tx=0x7fc5c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.980+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8037040 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.980+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8042070 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.980+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc0b27d0 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.980+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8030880 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.981+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc0cf090 con 0x7fc5dc15b980 2024-04-26T20:27:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.982+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0506c0 con 0x7fc5dc15b980 2024-04-26T20:27:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.983+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5c8030a20 con 0x7fc5dc15b980 2024-04-26T20:27:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.984+0000 7fc5d2a00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 0x7fc5bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.984+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fc5c80ac890 con 0x7fc5dc15b980 2024-04-26T20:27:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.984+0000 7fc5daa00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 0x7fc5bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.985+0000 7fc5daa00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 0x7fc5bc06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009560 tx=0x7fc5cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:26.988+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5c8045360 con 0x7fc5dc15b980 2024-04-26T20:27:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.200+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc5dc069560 con 0x7fc5dc15b980 2024-04-26T20:27:27.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.201+0000 7fc5d2a00640 1 -- 172.21.15.43:0/1040348277 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc5c807a7f0 con 0x7fc5dc15b980 2024-04-26T20:27:27.201 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 msgr2=0x7fc5bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 0x7fc5bc06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009560 tx=0x7fc5cc008040 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 msgr2=0x7fc5dc0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c00 tx=0x7fc5c80046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 shutdown_connections 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5bc06c920 0x7fc5bc06ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc15bd60 0x7fc5dc0d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5dc15b980 0x7fc5dc0d05c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 --2- 172.21.15.43:0/1040348277 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5dc0b4610 0x7fc5dc1527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 >> 172.21.15.43:0/1040348277 conn(0x7fc5dc068090 msgr2=0x7fc5dc0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 shutdown_connections 2024-04-26T20:27:27.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.204+0000 7fc5dbe00640 1 -- 172.21.15.43:0/1040348277 wait complete. 2024-04-26T20:27:27.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:27.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:27.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.436+0000 7f0415400640 1 Processor -- start 2024-04-26T20:27:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.436+0000 7f0415400640 1 -- start start 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f0415400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f04100b71a0 0x7f04100d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f0415400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04100b7580 0x7f041015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f0415400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04100d3a10 con 0x7f041015ef00 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f0415400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f04100c8830 con 0x7f04100b7580 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f0415400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f04100d3cb0 con 0x7f04100b71a0 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.437+0000 7f040f200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f041015ef00 0x7f0410162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36656/0 (socket says 172.21.15.43:36656) 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040f200640 1 -- 172.21.15.43:0/1670969606 learned_addr learned my addr 172.21.15.43:0/1670969606 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040de00640 1 --2- 172.21.15.43:0/1670969606 >> v2:172.21.15.43:3300/0 conn(0x7f04100b7580 0x7f041015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040e800640 1 --2- 172.21.15.43:0/1670969606 >> v2:172.21.15.164:3300/0 conn(0x7f04100b71a0 0x7f04100d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040d400640 1 -- 172.21.15.43:0/1670969606 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764785536 0 0) 0x7f04100d3a10 con 0x7f041015ef00 2024-04-26T20:27:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040d400640 1 -- 172.21.15.43:0/1670969606 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03f4003520 con 0x7f041015ef00 2024-04-26T20:27:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040de00640 1 -- 172.21.15.43:0/1670969606 >> v2:172.21.15.164:3300/0 conn(0x7f04100b71a0 msgr2=0x7f04100d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040de00640 1 --2- 172.21.15.43:0/1670969606 >> v2:172.21.15.164:3300/0 conn(0x7f04100b71a0 0x7f04100d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040de00640 1 -- 172.21.15.43:0/1670969606 >> v1:172.21.15.43:6789/0 conn(0x7f041015ef00 legacy=0x7f0410162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.438+0000 7f040de00640 1 -- 172.21.15.43:0/1670969606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04100c8af0 con 0x7f04100b7580 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.439+0000 7f040de00640 1 --2- 172.21.15.43:0/1670969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f041015e9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f041008d350 tx=0x7f040002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb437786a7ad6f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.440+0000 7f040d400640 1 -- 172.21.15.43:0/1670969606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400007740 con 0x7f04100b7580 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.440+0000 7f040d400640 1 -- 172.21.15.43:0/1670969606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04000078e0 con 0x7f04100b7580 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.440+0000 7f040d400640 1 -- 172.21.15.43:0/1670969606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400007cb0 con 0x7f04100b7580 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.440+0000 7f0415400640 1 -- 172.21.15.43:0/1670969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 msgr2=0x7f041015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.440+0000 7f0415400640 1 --2- 172.21.15.43:0/1670969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f041015e9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f041008d350 tx=0x7f040002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 -- 172.21.15.43:0/1670969606 shutdown_connections 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 --2- 172.21.15.43:0/1670969606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f041015e9c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 --2- 172.21.15.43:0/1670969606 >> v2:172.21.15.164:3300/0 conn(0x7f04100b71a0 0x7f04100d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 -- 172.21.15.43:0/1670969606 >> 172.21.15.43:0/1670969606 conn(0x7f04100d4d70 msgr2=0x7f04100d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 -- 172.21.15.43:0/1670969606 shutdown_connections 2024-04-26T20:27:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.441+0000 7f0415400640 1 -- 172.21.15.43:0/1670969606 wait complete. 2024-04-26T20:27:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.442+0000 7f0415400640 1 Processor -- start 2024-04-26T20:27:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 -- start start 2024-04-26T20:27:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f04100b71a0 0x7f04101f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 0x7f04100b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f041015f330 con 0x7f04100b7580 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04100c90b0 con 0x7f041015ef00 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.443+0000 7f0415400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f041015f4b0 con 0x7f04100b71a0 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32780/0 (socket says 172.21.15.43:32780) 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 -- 172.21.15.43:0/4216980366 learned_addr learned my addr 172.21.15.43:0/4216980366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040f200640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 0x7f04100b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 -- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f04100b71a0 msgr2=0x7f04101f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040e800640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f04100b71a0 0x7f04101f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f04100b71a0 0x7f04101f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 -- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 msgr2=0x7f04100b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.444+0000 7f040de00640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 0x7f04100b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f040de00640 1 -- 172.21.15.43:0/4216980366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04100cca00 con 0x7f04100b7580 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f040f200640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 0x7f04100b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f040de00640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f040002abe0 tx=0x7f0400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400040070 con 0x7f04100b7580 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0400031c40 con 0x7f04100b7580 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400009450 con 0x7f04100b7580 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04100ccce0 con 0x7f04100b7580 2024-04-26T20:27:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.445+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04100cd2a0 con 0x7f04100b7580 2024-04-26T20:27:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.447+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0410069350 con 0x7f04100b7580 2024-04-26T20:27:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.448+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0400039040 con 0x7f04100b7580 2024-04-26T20:27:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.449+0000 7f0406a00640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.449+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f04000ad470 con 0x7f04100b7580 2024-04-26T20:27:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.449+0000 7f040e800640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.450+0000 7f040e800640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009a10 tx=0x7f03f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.454+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f040007b3d0 con 0x7f04100b7580 2024-04-26T20:27:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.675+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f041008ee00 con 0x7f04100b7580 2024-04-26T20:27:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.676+0000 7f0406a00640 1 -- 172.21.15.43:0/4216980366 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f040007acd0 con 0x7f04100b7580 2024-04-26T20:27:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.678+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 msgr2=0x7f03ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.678+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009a10 tx=0x7f03f8008040 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.678+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 msgr2=0x7f04100b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.678+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f040002abe0 tx=0x7f0400004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 shutdown_connections 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f041015ef00 0x7f04100b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100b7580 0x7f04100b9370 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 --2- 172.21.15.43:0/4216980366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f04100b71a0 0x7f04101f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 >> 172.21.15.43:0/4216980366 conn(0x7f04100d4d70 msgr2=0x7f04100ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 shutdown_connections 2024-04-26T20:27:27.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.679+0000 7f0415400640 1 -- 172.21.15.43:0/4216980366 wait complete. 2024-04-26T20:27:27.700 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:27:27.700 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:27:27.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.973+0000 7f9482400640 1 Processor -- start 2024-04-26T20:27:27.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.973+0000 7f9482400640 1 -- start start 2024-04-26T20:27:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f9482400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f9482400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 0x7f947c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f9482400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f947c0d0480 con 0x7f947c0b50b0 2024-04-26T20:27:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f9482400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f947c0d0ce0 con 0x7f947c0d1a00 2024-04-26T20:27:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f9482400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f947c0d0720 con 0x7f947c0d1620 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.975+0000 7f947be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 0x7f947c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:48152/0 (socket says 172.21.15.43:48152) 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f947be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 0x7f947c0b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:56466/0 (socket says 172.21.15.43:56466) 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 -- 172.21.15.43:0/3054142428 learned_addr learned my addr 172.21.15.43:0/3054142428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 -- 172.21.15.43:0/3054142428 >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 msgr2=0x7f947c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 --2- 172.21.15.43:0/3054142428 >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 0x7f947c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 -- 172.21.15.43:0/3054142428 >> v1:172.21.15.164:6789/0 conn(0x7f947c0b50b0 legacy=0x7f947c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.976+0000 7f9481000640 1 -- 172.21.15.43:0/3054142428 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c0d0a00 con 0x7f947c0d1620 2024-04-26T20:27:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.978+0000 7f9481000640 1 --2- 172.21.15.43:0/3054142428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fcb44e33505471 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.978+0000 7f947b400640 1 -- 172.21.15.43:0/3054142428 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470037070 con 0x7f947c0d1620 2024-04-26T20:27:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.978+0000 7f947b400640 1 -- 172.21.15.43:0/3054142428 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9470032440 con 0x7f947c0d1620 2024-04-26T20:27:27.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.978+0000 7f947b400640 1 -- 172.21.15.43:0/3054142428 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470032810 con 0x7f947c0d1620 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 -- 172.21.15.43:0/3054142428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 msgr2=0x7f947c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 --2- 172.21.15.43:0/3054142428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 -- 172.21.15.43:0/3054142428 shutdown_connections 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 --2- 172.21.15.43:0/3054142428 >> v2:172.21.15.164:3300/0 conn(0x7f947c0d1a00 0x7f947c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 --2- 172.21.15.43:0/3054142428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c0d5750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 -- 172.21.15.43:0/3054142428 >> 172.21.15.43:0/3054142428 conn(0x7f947c0d4c30 msgr2=0x7f947c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 -- 172.21.15.43:0/3054142428 shutdown_connections 2024-04-26T20:27:27.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.979+0000 7f9482400640 1 -- 172.21.15.43:0/3054142428 wait complete. 2024-04-26T20:27:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.982+0000 7f9482400640 1 Processor -- start 2024-04-26T20:27:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.982+0000 7f9482400640 1 -- start start 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 0x7f947c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f947c0b5c50 con 0x7f947c0d1a00 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f947c0d1e60 con 0x7f947c0b50b0 2024-04-26T20:27:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.983+0000 7f9482400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f947c0b5dd0 con 0x7f947c0d1620 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32782/0 (socket says 172.21.15.43:32782) 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 -- 172.21.15.43:0/630540913 learned_addr learned my addr 172.21.15.43:0/630540913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481000640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 0x7f947c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f947be00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 -- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 msgr2=0x7f947c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 -- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 msgr2=0x7f947c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 0x7f947c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481a00640 1 -- 172.21.15.43:0/630540913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c0b77b0 con 0x7f947c0d1a00 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f947be00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.984+0000 7f9481000640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 0x7f947c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9481a00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f947400c7a0 tx=0x7f947400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474003b70 con 0x7f947c0d1a00 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474003d30 con 0x7f947c0d1a00 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c0c9e80 con 0x7f947c0d1a00 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947400aa90 con 0x7f947c0d1a00 2024-04-26T20:27:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.985+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c152930 con 0x7f947c0d1a00 2024-04-26T20:27:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.986+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947c069350 con 0x7f947c0d1a00 2024-04-26T20:27:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.988+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f947400ac30 con 0x7f947c0d1a00 2024-04-26T20:27:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.988+0000 7f9478c00640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 0x7f945c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.989+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f947408be20 con 0x7f947c0d1a00 2024-04-26T20:27:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.989+0000 7f9481000640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 0x7f945c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.990+0000 7f9481000640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 0x7f945c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9470034000 tx=0x7f947002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:27.993+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f947c0b77b0 con 0x7f947c0d1a00 2024-04-26T20:27:28.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.196+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f947c0b4cf0 con 0x7f947c0d1a00 2024-04-26T20:27:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.198+0000 7f9478c00640 1 -- 172.21.15.43:0/630540913 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f9474059e10 con 0x7f947c0d1a00 2024-04-26T20:27:28.199 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:28.199 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 msgr2=0x7f945c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 0x7f945c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9470034000 tx=0x7f947002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 msgr2=0x7f947c154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f947400c7a0 tx=0x7f947400cc60 comp rx=0 tx=0).stop 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 shutdown_connections 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f945c06c8f0 0x7f945c06edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f947c0d1620 0x7f947c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c0d1a00 0x7f947c154490 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 --2- 172.21.15.43:0/630540913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f947c0b50b0 0x7f947c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.200+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 >> 172.21.15.43:0/630540913 conn(0x7f947c0d4c30 msgr2=0x7f947c0d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.201+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 shutdown_connections 2024-04-26T20:27:28.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.201+0000 7f9482400640 1 -- 172.21.15.43:0/630540913 wait complete. 2024-04-26T20:27:28.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:28.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.436+0000 7fdf97e00640 1 Processor -- start 2024-04-26T20:27:28.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.437+0000 7fdf97e00640 1 -- start start 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.438+0000 7fdf97e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf98155cd0 0x7fdf98072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf97e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9815f920 con 0x7fdf98072b40 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf97e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9815fc00 con 0x7fdf980b5b10 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf97e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9815fee0 con 0x7fdf98155cd0 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf96000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdf98072b40 0x7fdf980b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36658/0 (socket says 172.21.15.43:36658) 2024-04-26T20:27:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf96000640 1 -- 172.21.15.43:0/1671901287 learned_addr learned my addr 172.21.15.43:0/1671901287 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf96a00640 1 --2- 172.21.15.43:0/1671901287 >> v2:172.21.15.164:3300/0 conn(0x7fdf98155cd0 0x7fdf98072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007889847 0 0) 0x7fdf9815f920 con 0x7fdf98072b40 2024-04-26T20:27:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.439+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf7c003520 con 0x7fdf98072b40 2024-04-26T20:27:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1220331833 0 0) 0x7fdf7c003520 con 0x7fdf98072b40 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 >> v2:172.21.15.164:3300/0 conn(0x7fdf98155cd0 msgr2=0x7fdf98072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 --2- 172.21.15.43:0/1671901287 >> v2:172.21.15.164:3300/0 conn(0x7fdf98155cd0 0x7fdf98072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 >> v1:172.21.15.164:6789/0 conn(0x7fdf980b5b10 legacy=0x7fdf98162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf981601c0 con 0x7fdf98072b40 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fdf84001670 con 0x7fdf98072b40 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 >> v1:172.21.15.43:6789/0 conn(0x7fdf98072b40 legacy=0x7fdf980b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf7c003a20 0x7fdf7c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c004800 0x7fdf7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 0x7fdf7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9815fee0 con 0x7fdf7c0051c0 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c00c0a0 con 0x7fdf7c004800 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c00c2a0 con 0x7fdf7c003a20 2024-04-26T20:27:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.440+0000 7fdf97400640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 0x7fdf7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf96000640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c004800 0x7fdf7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf96a00640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf7c003a20 0x7fdf7c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf97400640 1 -- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf7c003a20 msgr2=0x7fdf7c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf97400640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf7c003a20 0x7fdf7c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf97400640 1 -- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c004800 msgr2=0x7fdf7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf97400640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c004800 0x7fdf7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.441+0000 7fdf97400640 1 -- 172.21.15.43:0/1671901287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c003520 con 0x7fdf7c0051c0 2024-04-26T20:27:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.442+0000 7fdf97400640 1 -- 172.21.15.43:0/1671901287 reap_dead start 2024-04-26T20:27:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.442+0000 7fdf97400640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 0x7fdf7c00b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf90004480 tx=0x7fdf7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59f6a1df365c458 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.442+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004080 con 0x7fdf7c0051c0 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.442+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9002ad80 con 0x7fdf7c0051c0 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.442+0000 7fdf95600640 1 -- 172.21.15.43:0/1671901287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90006d90 con 0x7fdf7c0051c0 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 -- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 msgr2=0x7fdf7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 0x7fdf7c00b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf90004480 tx=0x7fdf7c005830 comp rx=0 tx=0).stop 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 -- 172.21.15.43:0/1671901287 shutdown_connections 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 --2- 172.21.15.43:0/1671901287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf7c0051c0 0x7fdf7c00b9c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 -- 172.21.15.43:0/1671901287 >> 172.21.15.43:0/1671901287 conn(0x7fdf98068090 msgr2=0x7fdf980684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 -- 172.21.15.43:0/1671901287 shutdown_connections 2024-04-26T20:27:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.443+0000 7fdf97e00640 1 -- 172.21.15.43:0/1671901287 wait complete. 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.445+0000 7fdf97e00640 1 Processor -- start 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.445+0000 7fdf97e00640 1 -- start start 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 0x7fdf981f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98161ea0 0x7fdf980b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98156230 con 0x7fdf981f59f0 2024-04-26T20:27:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981560b0 con 0x7fdf7c0051c0 2024-04-26T20:27:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf97e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9815f3f0 con 0x7fdf98161ea0 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf96000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32808/0 (socket says 172.21.15.43:32808) 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf96000640 1 -- 172.21.15.43:0/280708488 learned_addr learned my addr 172.21.15.43:0/280708488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.446+0000 7fdf96a00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 0x7fdf981f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf97400640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98161ea0 0x7fdf980b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 -- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98161ea0 msgr2=0x7fdf980b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98161ea0 0x7fdf980b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 -- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 msgr2=0x7fdf981f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 0x7fdf981f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 -- 172.21.15.43:0/280708488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf981f5e70 con 0x7fdf981f59f0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96a00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 0x7fdf981f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf96000640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e610 tx=0x7fdf8400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400c5f0 con 0x7fdf981f59f0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf84018070 con 0x7fdf981f59f0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400c9d0 con 0x7fdf981f59f0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.447+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf98159ac0 con 0x7fdf981f59f0 2024-04-26T20:27:28.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.448+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9815a080 con 0x7fdf981f59f0 2024-04-26T20:27:28.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.449+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9806de60 con 0x7fdf981f59f0 2024-04-26T20:27:28.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.450+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf8400cb70 con 0x7fdf981f59f0 2024-04-26T20:27:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.451+0000 7fdf8ea00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 0x7fdf7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.451+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fdf8408b990 con 0x7fdf981f59f0 2024-04-26T20:27:28.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.451+0000 7fdf96a00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 0x7fdf7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.452+0000 7fdf96a00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 0x7fdf7806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009ad0 tx=0x7fdf88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.455+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf9806de60 con 0x7fdf981f59f0 2024-04-26T20:27:28.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.661+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf980506c0 con 0x7fdf981f59f0 2024-04-26T20:27:28.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.662+0000 7fdf8ea00640 1 -- 172.21.15.43:0/280708488 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fdf981f5e70 con 0x7fdf981f59f0 2024-04-26T20:27:28.664 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:28.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.664+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 msgr2=0x7fdf7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.664+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 0x7fdf7806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009ad0 tx=0x7fdf88008040 comp rx=0 tx=0).stop 2024-04-26T20:27:28.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.664+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 msgr2=0x7fdf981f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.664+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400e610 tx=0x7fdf8400ead0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 shutdown_connections 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7806c920 0x7fdf7806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98161ea0 0x7fdf980b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf981f59f0 0x7fdf981f7730 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 --2- 172.21.15.43:0/280708488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf7c0051c0 0x7fdf981f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 >> 172.21.15.43:0/280708488 conn(0x7fdf98068090 msgr2=0x7fdf98068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 shutdown_connections 2024-04-26T20:27:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.665+0000 7fdf97e00640 1 -- 172.21.15.43:0/280708488 wait complete. 2024-04-26T20:27:28.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.896+0000 7fd766400640 1 Processor -- start 2024-04-26T20:27:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.896+0000 7fd766400640 1 -- start start 2024-04-26T20:27:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.897+0000 7fd766400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd76015cca0 0x7fd7600cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.897+0000 7fd766400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd7600d1f40 0x7fd7600d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd766400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7600d12e0 con 0x7fd760160240 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd766400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600d0240 con 0x7fd76015cca0 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd766400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd7600d0520 con 0x7fd7600d1f40 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75ea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd760160240 0x7fd7600d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75ea00640 1 -- 172.21.15.43:0/2264129667 learned_addr learned my addr 172.21.15.43:0/2264129667 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75f400640 1 --2- 172.21.15.43:0/2264129667 >> v2:172.21.15.43:3300/0 conn(0x7fd76015cca0 0x7fd7600cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75fe00640 1 --2- 172.21.15.43:0/2264129667 >> v2:172.21.15.164:3300/0 conn(0x7fd7600d1f40 0x7fd7600d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75f400640 1 -- 172.21.15.43:0/2264129667 >> v2:172.21.15.164:3300/0 conn(0x7fd7600d1f40 msgr2=0x7fd7600d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75f400640 1 --2- 172.21.15.43:0/2264129667 >> v2:172.21.15.164:3300/0 conn(0x7fd7600d1f40 0x7fd7600d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75f400640 1 -- 172.21.15.43:0/2264129667 >> v1:172.21.15.43:6789/0 conn(0x7fd760160240 legacy=0x7fd7600d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.899+0000 7fd75f400640 1 -- 172.21.15.43:0/2264129667 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600d0800 con 0x7fd76015cca0 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.898+0000 7fd75e000640 1 -- 172.21.15.43:0/2264129667 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3830952018 0 0) 0x7fd74c001110 con 0x7fd760160240 2024-04-26T20:27:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd75f400640 1 --2- 172.21.15.43:0/2264129667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76015cca0 0x7fd7600cfc30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cbea0 tx=0x7fd75002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7bb1c7a5be9403 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd75e000640 1 -- 172.21.15.43:0/2264129667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750037070 con 0x7fd76015cca0 2024-04-26T20:27:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd75e000640 1 -- 172.21.15.43:0/2264129667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd750032440 con 0x7fd76015cca0 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd75e000640 1 -- 172.21.15.43:0/2264129667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750032810 con 0x7fd76015cca0 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd766400640 1 -- 172.21.15.43:0/2264129667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76015cca0 msgr2=0x7fd7600cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.900+0000 7fd766400640 1 --2- 172.21.15.43:0/2264129667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76015cca0 0x7fd7600cfc30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cbea0 tx=0x7fd75002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 -- 172.21.15.43:0/2264129667 shutdown_connections 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 --2- 172.21.15.43:0/2264129667 >> v2:172.21.15.164:3300/0 conn(0x7fd7600d1f40 0x7fd7600d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 --2- 172.21.15.43:0/2264129667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd76015cca0 0x7fd7600cfc30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 -- 172.21.15.43:0/2264129667 >> 172.21.15.43:0/2264129667 conn(0x7fd7600d4e40 msgr2=0x7fd7600d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 -- 172.21.15.43:0/2264129667 shutdown_connections 2024-04-26T20:27:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.901+0000 7fd766400640 1 -- 172.21.15.43:0/2264129667 wait complete. 2024-04-26T20:27:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.902+0000 7fd766400640 1 Processor -- start 2024-04-26T20:27:28.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.903+0000 7fd766400640 1 -- start start 2024-04-26T20:27:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.903+0000 7fd766400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.903+0000 7fd766400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 0x7fd7600b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd766400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd760160240 0x7fd7600b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd766400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601606b0 con 0x7fd7600d1f40 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd766400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd7600d0dc0 con 0x7fd76015cca0 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd766400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd76015d080 con 0x7fd760160240 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32828/0 (socket says 172.21.15.43:32828) 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 -- 172.21.15.43:0/2605348103 learned_addr learned my addr 172.21.15.43:0/2605348103 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75ea00640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 0x7fd7600b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 -- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd760160240 msgr2=0x7fd7600b9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd760160240 0x7fd7600b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 -- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 msgr2=0x7fd7600b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 0x7fd7600b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75f400640 1 -- 172.21.15.43:0/2605348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7600cca00 con 0x7fd7600d1f40 2024-04-26T20:27:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.904+0000 7fd75ea00640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 0x7fd7600b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd75f400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd750035040 tx=0x7fd7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750037050 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd750038040 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750004470 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7600b8930 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.905+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7600b8ef0 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.906+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd760069350 con 0x7fd7600d1f40 2024-04-26T20:27:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.907+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd750030900 con 0x7fd7600d1f40 2024-04-26T20:27:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.908+0000 7fd757400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 0x7fd74006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.908+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd7500aca50 con 0x7fd7600d1f40 2024-04-26T20:27:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.908+0000 7fd75ea00640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 0x7fd74006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.909+0000 7fd75ea00640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 0x7fd74006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7600b9c10 tx=0x7fd74c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:28.913+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7600cca00 con 0x7fd7600d1f40 2024-04-26T20:27:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.116+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7600badc0 con 0x7fd7600d1f40 2024-04-26T20:27:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.117+0000 7fd757400640 1 -- 172.21.15.43:0/2605348103 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fd75007aa40 con 0x7fd7600d1f40 2024-04-26T20:27:29.119 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:29.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.119+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 msgr2=0x7fd74006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.119+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 0x7fd74006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7600b9c10 tx=0x7fd74c009040 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.119+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 msgr2=0x7fd76015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.119+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd750035040 tx=0x7fd7500046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 shutdown_connections 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd74006c8f0 0x7fd74006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd760160240 0x7fd7600b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd76015cca0 0x7fd7600b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 --2- 172.21.15.43:0/2605348103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7600d1f40 0x7fd76015a310 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 >> 172.21.15.43:0/2605348103 conn(0x7fd7600d4e40 msgr2=0x7fd76015c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 shutdown_connections 2024-04-26T20:27:29.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.120+0000 7fd766400640 1 -- 172.21.15.43:0/2605348103 wait complete. 2024-04-26T20:27:29.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.355+0000 7f5169200640 1 Processor -- start 2024-04-26T20:27:29.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.356+0000 7f5169200640 1 -- start start 2024-04-26T20:27:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.357+0000 7f5169200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 0x7f51640cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.357+0000 7f5169200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51640cfa00 0x7f51640b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.357+0000 7f5169200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51640ce9d0 con 0x7f516415d420 2024-04-26T20:27:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.357+0000 7f5169200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f51640cecb0 con 0x7f51640cfa00 2024-04-26T20:27:29.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5169200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51640b5bb0 con 0x7f51640d1b40 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5161e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f516415d420 0x7f51640cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36668/0 (socket says 172.21.15.43:36668) 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5161e00640 1 -- 172.21.15.43:0/2609243565 learned_addr learned my addr 172.21.15.43:0/2609243565 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5161400640 1 --2- 172.21.15.43:0/2609243565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 0x7f51640cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5162800640 1 --2- 172.21.15.43:0/2609243565 >> v2:172.21.15.164:3300/0 conn(0x7f51640cfa00 0x7f51640b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.358+0000 7f5160a00640 1 -- 172.21.15.43:0/2609243565 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891626973 0 0) 0x7f51640ce9d0 con 0x7f516415d420 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.359+0000 7f5160a00640 1 -- 172.21.15.43:0/2609243565 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5148003520 con 0x7f516415d420 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.359+0000 7f5161400640 1 -- 172.21.15.43:0/2609243565 >> v2:172.21.15.164:3300/0 conn(0x7f51640cfa00 msgr2=0x7f51640b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.359+0000 7f5161400640 1 --2- 172.21.15.43:0/2609243565 >> v2:172.21.15.164:3300/0 conn(0x7f51640cfa00 0x7f51640b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.359+0000 7f5161400640 1 -- 172.21.15.43:0/2609243565 >> v1:172.21.15.43:6789/0 conn(0x7f516415d420 legacy=0x7f51640cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.359+0000 7f5161400640 1 -- 172.21.15.43:0/2609243565 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51640b5e90 con 0x7f51640d1b40 2024-04-26T20:27:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.360+0000 7f5161400640 1 --2- 172.21.15.43:0/2609243565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 0x7f51640cd880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51500097a0 tx=0x7f515002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bbf58040f6119e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.360+0000 7f5160a00640 1 -- 172.21.15.43:0/2609243565 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150037070 con 0x7f51640d1b40 2024-04-26T20:27:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.360+0000 7f5160a00640 1 -- 172.21.15.43:0/2609243565 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5150032440 con 0x7f51640d1b40 2024-04-26T20:27:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.360+0000 7f5160a00640 1 -- 172.21.15.43:0/2609243565 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150032810 con 0x7f51640d1b40 2024-04-26T20:27:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 -- 172.21.15.43:0/2609243565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 msgr2=0x7f51640cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 --2- 172.21.15.43:0/2609243565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 0x7f51640cd880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51500097a0 tx=0x7f515002a180 comp rx=0 tx=0).stop 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 -- 172.21.15.43:0/2609243565 shutdown_connections 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 --2- 172.21.15.43:0/2609243565 >> v2:172.21.15.164:3300/0 conn(0x7f51640cfa00 0x7f51640b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 --2- 172.21.15.43:0/2609243565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640d1b40 0x7f51640cd880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.361+0000 7f5169200640 1 -- 172.21.15.43:0/2609243565 >> 172.21.15.43:0/2609243565 conn(0x7f51640d2bc0 msgr2=0x7f51640d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.362+0000 7f5169200640 1 -- 172.21.15.43:0/2609243565 shutdown_connections 2024-04-26T20:27:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.362+0000 7f5169200640 1 -- 172.21.15.43:0/2609243565 wait complete. 2024-04-26T20:27:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.363+0000 7f5169200640 1 Processor -- start 2024-04-26T20:27:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.364+0000 7f5169200640 1 -- start start 2024-04-26T20:27:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.364+0000 7f5169200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640cfa00 0x7f51641f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.364+0000 7f5169200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5169200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51640cbd30 0x7f516415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5169200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f51640d3830 con 0x7f516415d420 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5169200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51640d36b0 con 0x7f51640cbd30 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5169200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51640b6bd0 con 0x7f51640cfa00 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32838/0 (socket says 172.21.15.43:32838) 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 -- 172.21.15.43:0/2755355289 learned_addr learned my addr 172.21.15.43:0/2755355289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161e00640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640cfa00 0x7f51641f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5162800640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51640cbd30 0x7f516415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 -- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640cfa00 msgr2=0x7f51641f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640cfa00 0x7f51641f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 -- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51640cbd30 msgr2=0x7f516415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51640cbd30 0x7f516415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.365+0000 7f5161400640 1 -- 172.21.15.43:0/2755355289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641f6d80 con 0x7f516415d420 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f5161400640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51500098c0 tx=0x7f51500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150041030 con 0x7f516415d420 2024-04-26T20:27:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515003d4e0 con 0x7f516415d420 2024-04-26T20:27:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150037040 con 0x7f516415d420 2024-04-26T20:27:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641520d0 con 0x7f516415d420 2024-04-26T20:27:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.366+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51640ccab0 con 0x7f516415d420 2024-04-26T20:27:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.368+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5164069350 con 0x7f516415d420 2024-04-26T20:27:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.369+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5150044070 con 0x7f516415d420 2024-04-26T20:27:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.370+0000 7f515a000640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 0x7f514406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.370+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f51500ac640 con 0x7f516415d420 2024-04-26T20:27:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.370+0000 7f5161e00640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 0x7f514406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.371+0000 7f5161e00640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 0x7f514406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5154005880 tx=0x7f5154009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.374+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51641f6d80 con 0x7f516415d420 2024-04-26T20:27:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.576+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51640b4450 con 0x7f516415d420 2024-04-26T20:27:29.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.577+0000 7f515a000640 1 -- 172.21.15.43:0/2755355289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5150044e40 con 0x7f516415d420 2024-04-26T20:27:29.579 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 msgr2=0x7f514406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 0x7f514406edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5154005880 tx=0x7f5154009040 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 msgr2=0x7f516415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f51500098c0 tx=0x7f51500046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 shutdown_connections 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f514406c8f0 0x7f514406edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51640cbd30 0x7f516415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f516415d420 0x7f516415ee70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 --2- 172.21.15.43:0/2755355289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51640cfa00 0x7f51641f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.579+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 >> 172.21.15.43:0/2755355289 conn(0x7f51640d2bc0 msgr2=0x7f51640b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.580+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 shutdown_connections 2024-04-26T20:27:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.580+0000 7f5169200640 1 -- 172.21.15.43:0/2755355289 wait complete. 2024-04-26T20:27:29.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.805+0000 7f6e6f200640 1 Processor -- start 2024-04-26T20:27:29.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.805+0000 7f6e6f200640 1 -- start start 2024-04-26T20:27:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6f200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e7015bd60 0x7f6e70166830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6f200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7015c2a0 con 0x7f6e7015b980 2024-04-26T20:27:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7015c580 con 0x7f6e7006f3a0 2024-04-26T20:27:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6f200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7015d060 con 0x7f6e7015bd60 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6e800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e7015bd60 0x7f6e70166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6de00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6e7015b980 0x7f6e7006ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36670/0 (socket says 172.21.15.43:36670) 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.807+0000 7f6e6de00640 1 -- 172.21.15.43:0/2851374774 learned_addr learned my addr 172.21.15.43:0/2851374774 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.808+0000 7f6e6d400640 1 -- 172.21.15.43:0/2851374774 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2042301025 0 0) 0x7f6e7015c2a0 con 0x7f6e7015b980 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.808+0000 7f6e6d400640 1 -- 172.21.15.43:0/2851374774 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e54003520 con 0x7f6e7015b980 2024-04-26T20:27:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.808+0000 7f6e6e800640 1 -- 172.21.15.43:0/2851374774 >> v1:172.21.15.164:6789/0 conn(0x7f6e7006f3a0 legacy=0x7f6e700b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.808+0000 7f6e6e800640 1 -- 172.21.15.43:0/2851374774 >> v1:172.21.15.43:6789/0 conn(0x7f6e7015b980 legacy=0x7f6e7006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.808+0000 7f6e6e800640 1 -- 172.21.15.43:0/2851374774 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7015c860 con 0x7f6e7015bd60 2024-04-26T20:27:29.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.809+0000 7f6e6e800640 1 --2- 172.21.15.43:0/2851374774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e70166830 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cc7c71face5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.809+0000 7f6e6d400640 1 -- 172.21.15.43:0/2851374774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68007740 con 0x7f6e7015bd60 2024-04-26T20:27:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.809+0000 7f6e6d400640 1 -- 172.21.15.43:0/2851374774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e680078e0 con 0x7f6e7015bd60 2024-04-26T20:27:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.809+0000 7f6e6d400640 1 -- 172.21.15.43:0/2851374774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68007c90 con 0x7f6e7015bd60 2024-04-26T20:27:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 -- 172.21.15.43:0/2851374774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 msgr2=0x7f6e70166830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 --2- 172.21.15.43:0/2851374774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e70166830 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 -- 172.21.15.43:0/2851374774 shutdown_connections 2024-04-26T20:27:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 --2- 172.21.15.43:0/2851374774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e70166830 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 -- 172.21.15.43:0/2851374774 >> 172.21.15.43:0/2851374774 conn(0x7f6e70068090 msgr2=0x7f6e700684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 -- 172.21.15.43:0/2851374774 shutdown_connections 2024-04-26T20:27:29.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.810+0000 7f6e6f200640 1 -- 172.21.15.43:0/2851374774 wait complete. 2024-04-26T20:27:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.812+0000 7f6e6f200640 1 Processor -- start 2024-04-26T20:27:29.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.812+0000 7f6e6f200640 1 -- start start 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 0x7f6e70150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e7015b980 0x7f6e7014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b4d20 con 0x7f6e7015bd60 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b52b0 con 0x7f6e7015b980 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700b4ea0 con 0x7f6e7006f3a0 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32866/0 (socket says 172.21.15.43:32866) 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.813+0000 7f6e6e800640 1 -- 172.21.15.43:0/3259499638 learned_addr learned my addr 172.21.15.43:0/3259499638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6de00640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 0x7f6e70150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 -- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 msgr2=0x7f6e70150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 0x7f6e70150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 -- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e7015b980 msgr2=0x7f6e7014d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e67e00640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e7015b980 0x7f6e7014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e7015b980 0x7f6e7014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 -- 172.21.15.43:0/3259499638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7014de20 con 0x7f6e7015bd60 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6de00640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 0x7f6e70150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6e800640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6803f070 con 0x7f6e7015bd60 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68031c40 con 0x7f6e7015bd60 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68009450 con 0x7f6e7015bd60 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.814+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7014e0c0 con 0x7f6e7015bd60 2024-04-26T20:27:29.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.815+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e701505c0 con 0x7f6e7015bd60 2024-04-26T20:27:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.816+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e700506c0 con 0x7f6e7015bd60 2024-04-26T20:27:29.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.817+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e68003990 con 0x7f6e7015bd60 2024-04-26T20:27:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.818+0000 7f6e66000640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.818+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f6e680ad070 con 0x7f6e7015bd60 2024-04-26T20:27:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.818+0000 7f6e6de00640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.819+0000 7f6e6de00640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e600011e0 tx=0x7f6e60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:29.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:29.823+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e6807b060 con 0x7f6e7015bd60 2024-04-26T20:27:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.019+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6e7006cb40 con 0x7f6e7015bd60 2024-04-26T20:27:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.020+0000 7f6e66000640 1 -- 172.21.15.43:0/3259499638 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6e6807a960 con 0x7f6e7015bd60 2024-04-26T20:27:30.022 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.022+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 msgr2=0x7f6e4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.022+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e600011e0 tx=0x7f6e60009040 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.022+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 msgr2=0x7f6e7014d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.022+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e68004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 shutdown_connections 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6e4c06c8f0 0x7f6e4c06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7015bd60 0x7f6e7014d700 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e7015b980 0x7f6e7014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 --2- 172.21.15.43:0/3259499638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e7006f3a0 0x7f6e70150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 >> 172.21.15.43:0/3259499638 conn(0x7f6e70068090 msgr2=0x7f6e7015d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:30.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 shutdown_connections 2024-04-26T20:27:30.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.023+0000 7f6e6f200640 1 -- 172.21.15.43:0/3259499638 wait complete. 2024-04-26T20:27:30.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:27:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.240+0000 7f5f26000640 1 Processor -- start 2024-04-26T20:27:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.241+0000 7f5f26000640 1 -- start start 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.242+0000 7f5f26000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f200d1620 0x7f5f200d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f26000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f200d0480 con 0x7f5f200d1a00 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f26000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f200d0720 con 0x7f5f200d3810 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f26000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5f200d0ce0 con 0x7f5f200d1620 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f1fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5f200d1a00 0x7f5f200b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36672/0 (socket says 172.21.15.43:36672) 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f1fe00640 1 -- 172.21.15.43:0/4172386817 learned_addr learned my addr 172.21.15.43:0/4172386817 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f24c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f200d1620 0x7f5f200d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f24c00640 1 -- 172.21.15.43:0/4172386817 >> v1:172.21.15.164:6789/0 conn(0x7f5f200d3810 legacy=0x7f5f200b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f1f400640 1 -- 172.21.15.43:0/4172386817 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461139523 0 0) 0x7f5f200d0480 con 0x7f5f200d1a00 2024-04-26T20:27:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.243+0000 7f5f24c00640 1 -- 172.21.15.43:0/4172386817 >> v1:172.21.15.43:6789/0 conn(0x7f5f200d1a00 legacy=0x7f5f200b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.244+0000 7f5f24c00640 1 -- 172.21.15.43:0/4172386817 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f200d0a00 con 0x7f5f200d1620 2024-04-26T20:27:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.245+0000 7f5f24c00640 1 --2- 172.21.15.43:0/4172386817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d1620 0x7f5f200d5750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f1002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763f5bf49a92339 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.245+0000 7f5f1f400640 1 -- 172.21.15.43:0/4172386817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10031600 con 0x7f5f200d1620 2024-04-26T20:27:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.245+0000 7f5f1f400640 1 -- 172.21.15.43:0/4172386817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f100317a0 con 0x7f5f200d1620 2024-04-26T20:27:30.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.245+0000 7f5f1f400640 1 -- 172.21.15.43:0/4172386817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10031b70 con 0x7f5f200d1620 2024-04-26T20:27:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 -- 172.21.15.43:0/4172386817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d1620 msgr2=0x7f5f200d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 --2- 172.21.15.43:0/4172386817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d1620 0x7f5f200d5750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f1002a610 comp rx=0 tx=0).stop 2024-04-26T20:27:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 -- 172.21.15.43:0/4172386817 shutdown_connections 2024-04-26T20:27:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 --2- 172.21.15.43:0/4172386817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d1620 0x7f5f200d5750 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 -- 172.21.15.43:0/4172386817 >> 172.21.15.43:0/4172386817 conn(0x7f5f200d4c30 msgr2=0x7f5f200d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 -- 172.21.15.43:0/4172386817 shutdown_connections 2024-04-26T20:27:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.246+0000 7f5f26000640 1 -- 172.21.15.43:0/4172386817 wait complete. 2024-04-26T20:27:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.248+0000 7f5f26000640 1 Processor -- start 2024-04-26T20:27:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.248+0000 7f5f26000640 1 -- start start 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 0x7f5f2015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f200d1a00 0x7f5f200c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200b5990 con 0x7f5f200d3810 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f200d1e60 con 0x7f5f200d1620 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f26000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5f200b5b10 con 0x7f5f200d1a00 2024-04-26T20:27:30.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f25600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f25600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32880/0 (socket says 172.21.15.43:32880) 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f25600640 1 -- 172.21.15.43:0/4015656129 learned_addr learned my addr 172.21.15.43:0/4015656129 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f1fe00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f200d1a00 0x7f5f200c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.249+0000 7f5f24c00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 0x7f5f2015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 -- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f200d1a00 msgr2=0x7f5f200c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f200d1a00 0x7f5f200c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 -- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 msgr2=0x7f5f2015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 0x7f5f2015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 -- 172.21.15.43:0/4015656129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f201f4ea0 con 0x7f5f200d3810 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f24c00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 0x7f5f2015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f25600640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b410 tx=0x7f5f1800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18015030 con 0x7f5f200d3810 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f18004020 con 0x7f5f200d3810 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004450 con 0x7f5f200d3810 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f201f5180 con 0x7f5f200d3810 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.250+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f200c8ac0 con 0x7f5f200d3810 2024-04-26T20:27:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.252+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f200c8cc0 con 0x7f5f200d3810 2024-04-26T20:27:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.252+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f18018070 con 0x7f5f200d3810 2024-04-26T20:27:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.253+0000 7f5f1cc00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 0x7f5f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.254+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f5f1808ba70 con 0x7f5f200d3810 2024-04-26T20:27:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.254+0000 7f5f24c00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 0x7f5f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.255+0000 7f5f24c00640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 0x7f5f0006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.258+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f200c8cc0 con 0x7f5f200d3810 2024-04-26T20:27:30.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.484+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5f200c8590 con 0x7f5f200d3810 2024-04-26T20:27:30.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.485+0000 7f5f1cc00640 1 -- 172.21.15.43:0/4015656129 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f5f18059a60 con 0x7f5f200d3810 2024-04-26T20:27:30.486 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 msgr2=0x7f5f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 0x7f5f0006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002f000 comp rx=0 tx=0).stop 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 msgr2=0x7f5f201f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b410 tx=0x7f5f1800b8d0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 shutdown_connections 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5f0006c8f0 0x7f5f0006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f200d1a00 0x7f5f200c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f200d3810 0x7f5f201f43b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.487+0000 7f5f26000640 1 --2- 172.21.15.43:0/4015656129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f200d1620 0x7f5f2015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.488+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 >> 172.21.15.43:0/4015656129 conn(0x7f5f200d4c30 msgr2=0x7f5f200cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.488+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 shutdown_connections 2024-04-26T20:27:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.488+0000 7f5f26000640 1 -- 172.21.15.43:0/4015656129 wait complete. 2024-04-26T20:27:30.508 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2024-04-26T20:27:30.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:30.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-26T20:27:30.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:30.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:27:30.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:27:30.597 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T20:27:30.653 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:27:30.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:27:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.863+0000 7faa7ec00640 1 Processor -- start 2024-04-26T20:27:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.863+0000 7faa7ec00640 1 -- start start 2024-04-26T20:27:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.864+0000 7faa7ec00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faa7806c350 0x7faa7806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.864+0000 7faa7ec00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa780734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa7ec00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7815fb10 con 0x7faa78159720 2024-04-26T20:27:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa7ec00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7faa78160440 con 0x7faa7806c350 2024-04-26T20:27:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa7ec00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa7815fdf0 con 0x7faa78073090 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa77400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faa7806c350 0x7faa7806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa77400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faa7806c350 0x7faa7806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.43:56554/0 (socket says 172.21.15.43:56554) 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.865+0000 7faa77400640 1 -- 172.21.15.43:0/3327652611 learned_addr learned my addr 172.21.15.43:0/3327652611 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa7cc00640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa780734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa77400640 1 -- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 msgr2=0x7faa780734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa77400640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa780734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa77400640 1 -- 172.21.15.43:0/3327652611 >> v1:172.21.15.164:6789/0 conn(0x7faa78159720 legacy=0x7faa780b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa77400640 1 -- 172.21.15.43:0/3327652611 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781600d0 con 0x7faa7806c350 2024-04-26T20:27:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.866+0000 7faa7cc00640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa780734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.867+0000 7faa77400640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa7806c800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa6402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aaaee322d4e15b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.868+0000 7faa76a00640 1 -- 172.21.15.43:0/3327652611 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64037070 con 0x7faa7806c350 2024-04-26T20:27:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.868+0000 7faa76a00640 1 -- 172.21.15.43:0/3327652611 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa64032440 con 0x7faa7806c350 2024-04-26T20:27:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.868+0000 7faa76a00640 1 -- 172.21.15.43:0/3327652611 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64032810 con 0x7faa7806c350 2024-04-26T20:27:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.868+0000 7faa7ec00640 1 -- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 msgr2=0x7faa7806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.868+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa7806c800 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa6402a140 comp rx=0 tx=0).stop 2024-04-26T20:27:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 -- 172.21.15.43:0/3327652611 shutdown_connections 2024-04-26T20:27:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa780734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3327652611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa7806c800 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 -- 172.21.15.43:0/3327652611 >> 172.21.15.43:0/3327652611 conn(0x7faa780b39b0 msgr2=0x7faa780b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 -- 172.21.15.43:0/3327652611 shutdown_connections 2024-04-26T20:27:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.869+0000 7faa7ec00640 1 -- 172.21.15.43:0/3327652611 wait complete. 2024-04-26T20:27:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.870+0000 7faa7ec00640 1 Processor -- start 2024-04-26T20:27:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.871+0000 7faa7ec00640 1 -- start start 2024-04-26T20:27:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa78144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa781ea970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa780b6e20 con 0x7faa78159720 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa7815f7c0 con 0x7faa7806c350 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7ec00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa78159b00 con 0x7faa78073090 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32894/0 (socket says 172.21.15.43:32894) 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 -- 172.21.15.43:0/3815193524 learned_addr learned my addr 172.21.15.43:0/3815193524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa77e00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa78144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa77400640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa781ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 -- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 msgr2=0x7faa781ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa781ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 -- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 msgr2=0x7faa78144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa78144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa7cc00640 1 -- 172.21.15.43:0/3815193524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781e80b0 con 0x7faa78159720 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.872+0000 7faa77e00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa78144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa7cc00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa70007f20 tx=0x7faa7000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7001a070 con 0x7faa78159720 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa70003b90 con 0x7faa78159720 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70004030 con 0x7faa78159720 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa780d16a0 con 0x7faa78159720 2024-04-26T20:27:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.873+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa78145680 con 0x7faa78159720 2024-04-26T20:27:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.874+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78076260 con 0x7faa78159720 2024-04-26T20:27:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.875+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa7001d070 con 0x7faa78159720 2024-04-26T20:27:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.876+0000 7faa6fe00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 0x7faa5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.876+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7faa7008c2b0 con 0x7faa78159720 2024-04-26T20:27:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.877+0000 7faa77e00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 0x7faa5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.877+0000 7faa77e00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 0x7faa5806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa780b4ed0 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:30.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:30.881+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa78076260 con 0x7faa78159720 2024-04-26T20:27:31.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.125+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa78069350 con 0x7faa78159720 2024-04-26T20:27:31.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.126+0000 7faa6fe00640 1 -- 172.21.15.43:0/3815193524 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7faa781e80b0 con 0x7faa78159720 2024-04-26T20:27:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:31.128 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:27:31.128 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:27:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.128+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 msgr2=0x7faa5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.128+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 0x7faa5806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faa780b4ed0 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-04-26T20:27:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.128+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 msgr2=0x7faa781eaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.128+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa70007f20 tx=0x7faa7000d560 comp rx=0 tx=0).stop 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 shutdown_connections 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa5806c8f0 0x7faa5806edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa78073090 0x7faa781ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa78159720 0x7faa781eaeb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 --2- 172.21.15.43:0/3815193524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa7806c350 0x7faa78144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 >> 172.21.15.43:0/3815193524 conn(0x7faa780b39b0 msgr2=0x7faa780b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 shutdown_connections 2024-04-26T20:27:31.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.129+0000 7faa7ec00640 1 -- 172.21.15.43:0/3815193524 wait complete. 2024-04-26T20:27:31.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.360+0000 7f3bb3200640 1 Processor -- start 2024-04-26T20:27:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.360+0000 7f3bb3200640 1 -- start start 2024-04-26T20:27:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.361+0000 7f3bb3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3bac167b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb3200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac15d3d0 con 0x7f3bac157430 2024-04-26T20:27:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb3200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac15d6b0 con 0x7f3bac06ca60 2024-04-26T20:27:31.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0d1360 con 0x7f3bac0b0ff0 2024-04-26T20:27:31.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb1e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3bac157430 0x7f3bac06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36674/0 (socket says 172.21.15.43:36674) 2024-04-26T20:27:31.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb1e00640 1 -- 172.21.15.43:0/2295839092 learned_addr learned my addr 172.21.15.43:0/2295839092 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.362+0000 7f3bb2800640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3bac167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253222807 0 0) 0x7f3bac15d3d0 con 0x7f3bac157430 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b94003520 con 0x7f3bac157430 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266776206 0 0) 0x7f3bac15d6b0 con 0x7f3bac06ca60 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bac15d3d0 con 0x7f3bac06ca60 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2026674240 0 0) 0x7f3b94003520 con 0x7f3bac157430 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 msgr2=0x7f3bac167b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3bac167b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 >> v1:172.21.15.164:6789/0 conn(0x7f3bac06ca60 legacy=0x7f3bac0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac15d990 con 0x7f3bac157430 2024-04-26T20:27:31.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f3ba0001670 con 0x7f3bac157430 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 >> v1:172.21.15.43:6789/0 conn(0x7f3bac157430 legacy=0x7f3bac06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3b94003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b94004120 0x7f3b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9400bb50 con 0x7f3b94004120 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9400bd00 con 0x7f3b94004d40 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.363+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9400bf00 con 0x7f3bac0b0ff0 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b94004120 0x7f3b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1e00640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3b94003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb2800640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 -- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 msgr2=0x7f3b94003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:31.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac0b0ff0 0x7f3b94003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 -- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004d40 msgr2=0x7f3b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004d40 0x7f3b940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.364+0000 7f3bb1400640 1 -- 172.21.15.43:0/2295839092 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b94003520 con 0x7f3b94004120 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.365+0000 7f3bb1400640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b94004120 0x7f3b94004800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c001fd0 tx=0x7f3b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d971199a3acee5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c001bf0 con 0x7f3b94004120 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9c003b70 con 0x7f3b94004120 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb0a00640 1 -- 172.21.15.43:0/2295839092 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0050d0 con 0x7f3b94004120 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 -- 172.21.15.43:0/2295839092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b94004120 msgr2=0x7f3b94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2295839092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b94004120 0x7f3b94004800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c001fd0 tx=0x7f3b94005830 comp rx=0 tx=0).stop 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb2800640 1 -- 172.21.15.43:0/2295839092 reap_dead start 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 -- 172.21.15.43:0/2295839092 shutdown_connections 2024-04-26T20:27:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 -- 172.21.15.43:0/2295839092 >> 172.21.15.43:0/2295839092 conn(0x7f3bac075df0 msgr2=0x7f3bac068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 -- 172.21.15.43:0/2295839092 shutdown_connections 2024-04-26T20:27:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.366+0000 7f3bb3200640 1 -- 172.21.15.43:0/2295839092 wait complete. 2024-04-26T20:27:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.368+0000 7f3bb3200640 1 Processor -- start 2024-04-26T20:27:31.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.368+0000 7f3bb3200640 1 -- start start 2024-04-26T20:27:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.369+0000 7f3bb3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004120 0x7f3bac0af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.369+0000 7f3bb3200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac15ed70 0x7f3bac14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.369+0000 7f3bb3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.369+0000 7f3bb3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0d0dd0 con 0x7f3bac153210 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.369+0000 7f3bb3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac15df50 con 0x7f3b94004120 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb3200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0d0f50 con 0x7f3bac15ed70 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32914/0 (socket says 172.21.15.43:32914) 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 -- 172.21.15.43:0/2759474831 learned_addr learned my addr 172.21.15.43:0/2759474831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb1400640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac15ed70 0x7f3bac14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 -- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac15ed70 msgr2=0x7f3bac14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac15ed70 0x7f3bac14b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 -- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004120 msgr2=0x7f3bac0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004120 0x7f3bac0af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.370+0000 7f3bb2800640 1 -- 172.21.15.43:0/2759474831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac15f1f0 con 0x7f3bac153210 2024-04-26T20:27:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3bb2800640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0b1440 tx=0x7f3ba8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8019020 con 0x7f3bac153210 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba8003610 con 0x7f3bac153210 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8003a00 con 0x7f3bac153210 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac15e500 con 0x7f3bac153210 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.371+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac14b780 con 0x7f3bac153210 2024-04-26T20:27:31.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.372+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac069350 con 0x7f3bac153210 2024-04-26T20:27:31.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.373+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ba801c020 con 0x7f3bac153210 2024-04-26T20:27:31.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.375+0000 7f3ba6000640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.375+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f3ba808c0e0 con 0x7f3bac153210 2024-04-26T20:27:31.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.375+0000 7f3bb1e00640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.376+0000 7f3bb1e00640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 0x7f3b9006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001eb0 tx=0x7f3ba0000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.380+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3bac069350 con 0x7f3bac153210 2024-04-26T20:27:31.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.588+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bac069a60 con 0x7f3bac153210 2024-04-26T20:27:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.591+0000 7f3ba6000640 1 -- 172.21.15.43:0/2759474831 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f3ba8023030 con 0x7f3bac153210 2024-04-26T20:27:31.593 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:31.594 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:31.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.594+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 msgr2=0x7f3b9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.594+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 0x7f3b9006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001eb0 tx=0x7f3ba0000ed0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.594+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 msgr2=0x7f3bac0b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.594+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0b1440 tx=0x7f3ba8011920 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.594+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 shutdown_connections 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3b9006c920 0x7f3b9006ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bac15ed70 0x7f3bac14b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac153210 0x7f3bac0b24c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 --2- 172.21.15.43:0/2759474831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3b94004120 0x7f3bac0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 >> 172.21.15.43:0/2759474831 conn(0x7f3bac075df0 msgr2=0x7f3bac0732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 shutdown_connections 2024-04-26T20:27:31.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.595+0000 7f3bb3200640 1 -- 172.21.15.43:0/2759474831 wait complete. 2024-04-26T20:27:31.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:25:54.942670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:25:56.403597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:26:00.033245+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:27:31.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.833+0000 7fd9a7200640 1 Processor -- start 2024-04-26T20:27:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.833+0000 7fd9a7200640 1 -- start start 2024-04-26T20:27:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a815f120 0x7fd9a80d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a7200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 0x7fd9a80d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a7200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a80d0c40 con 0x7fd9a80d37a0 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a80d14c0 con 0x7fd9a815f120 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a7200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a80d3bd0 con 0x7fd9a80d2b80 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a5e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a815f120 0x7fd9a80d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a5e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9a815f120 0x7fd9a80d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32930/0 (socket says 172.21.15.43:32930) 2024-04-26T20:27:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.835+0000 7fd9a5e00640 1 -- 172.21.15.43:0/3437829604 learned_addr learned my addr 172.21.15.43:0/3437829604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5400640 1 --2- 172.21.15.43:0/3437829604 >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 0x7fd9a80d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5e00640 1 -- 172.21.15.43:0/3437829604 >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 msgr2=0x7fd9a80d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/3437829604 >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 0x7fd9a80d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5e00640 1 -- 172.21.15.43:0/3437829604 >> v1:172.21.15.164:6789/0 conn(0x7fd9a80d37a0 legacy=0x7fd9a80d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5e00640 1 -- 172.21.15.43:0/3437829604 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a80d0f00 con 0x7fd9a815f120 2024-04-26T20:27:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.836+0000 7fd9a5400640 1 --2- 172.21.15.43:0/3437829604 >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 0x7fd9a80d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.837+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/3437829604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a815f120 0x7fd9a80d0610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd9a808d7c0 tx=0x7fd99802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b457872f6b7d2868 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.837+0000 7fd9a4a00640 1 -- 172.21.15.43:0/3437829604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998037070 con 0x7fd9a815f120 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.837+0000 7fd9a4a00640 1 -- 172.21.15.43:0/3437829604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd998032440 con 0x7fd9a815f120 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.837+0000 7fd9a4a00640 1 -- 172.21.15.43:0/3437829604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998032810 con 0x7fd9a815f120 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 -- 172.21.15.43:0/3437829604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a815f120 msgr2=0x7fd9a80d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 --2- 172.21.15.43:0/3437829604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a815f120 0x7fd9a80d0610 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd9a808d7c0 tx=0x7fd99802a230 comp rx=0 tx=0).stop 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 -- 172.21.15.43:0/3437829604 shutdown_connections 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 --2- 172.21.15.43:0/3437829604 >> v2:172.21.15.164:3300/0 conn(0x7fd9a80d2b80 0x7fd9a80d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 --2- 172.21.15.43:0/3437829604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a815f120 0x7fd9a80d0610 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 -- 172.21.15.43:0/3437829604 >> 172.21.15.43:0/3437829604 conn(0x7fd9a80d4d10 msgr2=0x7fd9a80d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 -- 172.21.15.43:0/3437829604 shutdown_connections 2024-04-26T20:27:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.838+0000 7fd9a7200640 1 -- 172.21.15.43:0/3437829604 wait complete. 2024-04-26T20:27:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.840+0000 7fd9a7200640 1 Processor -- start 2024-04-26T20:27:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.840+0000 7fd9a7200640 1 -- start start 2024-04-26T20:27:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 0x7fd9a8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 0x7fd9a8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a815f500 con 0x7fd9a80d37a0 2024-04-26T20:27:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80d2fc0 con 0x7fd9a80d2b80 2024-04-26T20:27:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a815f680 con 0x7fd9a815f120 2024-04-26T20:27:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.841+0000 7fd9a5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32946/0 (socket says 172.21.15.43:32946) 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 -- 172.21.15.43:0/2155147974 learned_addr learned my addr 172.21.15.43:0/2155147974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 0x7fd9a8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a6800640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 0x7fd9a8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 -- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 msgr2=0x7fd9a8154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 0x7fd9a8154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 -- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 msgr2=0x7fd9a8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 0x7fd9a8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 -- 172.21.15.43:0/2155147974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a80b8bd0 con 0x7fd9a80d37a0 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 0x7fd9a8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a6800640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 0x7fd9a8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.842+0000 7fd9a5400640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c7f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.843+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994015070 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.843+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994007820 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.843+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a80b8eb0 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.843+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994007bd0 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.843+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a815a430 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.844+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a808da00 con 0x7fd9a80d37a0 2024-04-26T20:27:31.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.845+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd994018070 con 0x7fd9a80d37a0 2024-04-26T20:27:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.846+0000 7fd99e000640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 0x7fd98806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.847+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fd99408c060 con 0x7fd9a80d37a0 2024-04-26T20:27:31.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.847+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 0x7fd98806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:31.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.848+0000 7fd9a5e00640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 0x7fd98806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd998034000 tx=0x7fd99802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:31.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:31.851+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9a808da00 con 0x7fd9a80d37a0 2024-04-26T20:27:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.055+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9a80506c0 con 0x7fd9a80d37a0 2024-04-26T20:27:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.056+0000 7fd99e000640 1 -- 172.21.15.43:0/2155147974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd99405a0f0 con 0x7fd9a80d37a0 2024-04-26T20:27:32.058 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.059+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 msgr2=0x7fd98806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.059+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 0x7fd98806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd998034000 tx=0x7fd99802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.059+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 msgr2=0x7fd9a80b8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.059+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c7f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 shutdown_connections 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd98806c8f0 0x7fd98806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9a815f120 0x7fd9a8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a80d37a0 0x7fd9a80b8430 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 --2- 172.21.15.43:0/2155147974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9a80d2b80 0x7fd9a8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 >> 172.21.15.43:0/2155147974 conn(0x7fd9a80d4d10 msgr2=0x7fd9a81622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 shutdown_connections 2024-04-26T20:27:32.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.060+0000 7fd9a7200640 1 -- 172.21.15.43:0/2155147974 wait complete. 2024-04-26T20:27:32.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:27:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.297+0000 7f3633a00640 1 Processor -- start 2024-04-26T20:27:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.297+0000 7f3633a00640 1 -- start start 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.298+0000 7f3633a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f362c0d3780 0x7f362c0d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.298+0000 7f3633a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f362c0d2d70 0x7f362c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.298+0000 7f3633a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f362c0d40a0 con 0x7f362c0d31d0 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.298+0000 7f3633a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f362c0b5d70 con 0x7f362c0d3780 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.298+0000 7f3633a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f362c0b5160 con 0x7f362c0d2d70 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f362c0d3780 0x7f362c0d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f362c0d3780 0x7f362c0d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32956/0 (socket says 172.21.15.43:32956) 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 -- 172.21.15.43:0/2276466716 learned_addr learned my addr 172.21.15.43:0/2276466716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f362be00640 1 --2- 172.21.15.43:0/2276466716 >> v2:172.21.15.164:3300/0 conn(0x7f362c0d2d70 0x7f362c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 -- 172.21.15.43:0/2276466716 >> v2:172.21.15.164:3300/0 conn(0x7f362c0d2d70 msgr2=0x7f362c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:32.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 --2- 172.21.15.43:0/2276466716 >> v2:172.21.15.164:3300/0 conn(0x7f362c0d2d70 0x7f362c0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 -- 172.21.15.43:0/2276466716 >> v1:172.21.15.43:6789/0 conn(0x7f362c0d31d0 legacy=0x7f362c0d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.299+0000 7f3631000640 1 -- 172.21.15.43:0/2276466716 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c0b5440 con 0x7f362c0d3780 2024-04-26T20:27:32.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.300+0000 7f3631000640 1 --2- 172.21.15.43:0/2276466716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d3780 0x7f362c0d3b60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f362c0b7d70 tx=0x7f361c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ed589574f37841 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.301+0000 7f362b400640 1 -- 172.21.15.43:0/2276466716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c036070 con 0x7f362c0d3780 2024-04-26T20:27:32.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.301+0000 7f362b400640 1 -- 172.21.15.43:0/2276466716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c031600 con 0x7f362c0d3780 2024-04-26T20:27:32.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.301+0000 7f362b400640 1 -- 172.21.15.43:0/2276466716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c0319d0 con 0x7f362c0d3780 2024-04-26T20:27:32.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.301+0000 7f3633a00640 1 -- 172.21.15.43:0/2276466716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d3780 msgr2=0x7f362c0d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.301+0000 7f3633a00640 1 --2- 172.21.15.43:0/2276466716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d3780 0x7f362c0d3b60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f362c0b7d70 tx=0x7f361c02a730 comp rx=0 tx=0).stop 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 -- 172.21.15.43:0/2276466716 shutdown_connections 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 --2- 172.21.15.43:0/2276466716 >> v2:172.21.15.164:3300/0 conn(0x7f362c0d2d70 0x7f362c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 --2- 172.21.15.43:0/2276466716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d3780 0x7f362c0d3b60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 -- 172.21.15.43:0/2276466716 >> 172.21.15.43:0/2276466716 conn(0x7f362c08ec50 msgr2=0x7f362c09e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 -- 172.21.15.43:0/2276466716 shutdown_connections 2024-04-26T20:27:32.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.302+0000 7f3633a00640 1 -- 172.21.15.43:0/2276466716 wait complete. 2024-04-26T20:27:32.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.304+0000 7f3633a00640 1 Processor -- start 2024-04-26T20:27:32.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.304+0000 7f3633a00640 1 -- start start 2024-04-26T20:27:32.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f362c0d31d0 0x7f362c0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f362c0d3780 0x7f362c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f362c15f5e0 con 0x7f362c0d2d70 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f362c0b5a00 con 0x7f362c0d31d0 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3633a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f362c0b4920 con 0x7f362c0d3780 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32964/0 (socket says 172.21.15.43:32964) 2024-04-26T20:27:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 -- 172.21.15.43:0/4048719398 learned_addr learned my addr 172.21.15.43:0/4048719398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f362be00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f362c0d31d0 0x7f362c0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 -- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f362c0d3780 msgr2=0x7f362c0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f362c0d3780 0x7f362c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.306+0000 7f3631000640 1 -- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f362c0d31d0 msgr2=0x7f362c0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3631000640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f362c0d31d0 0x7f362c0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3631000640 1 -- 172.21.15.43:0/4048719398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c159540 con 0x7f362c0d2d70 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3631000640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f361c000f80 tx=0x7f361c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c036050 con 0x7f362c0d2d70 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c042070 con 0x7f362c0d2d70 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c009510 con 0x7f362c0d2d70 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c159a60 con 0x7f362c0d2d70 2024-04-26T20:27:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.307+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c0b8930 con 0x7f362c0d2d70 2024-04-26T20:27:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.309+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c069350 con 0x7f362c0d2d70 2024-04-26T20:27:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.310+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f361c007c70 con 0x7f362c0d2d70 2024-04-26T20:27:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.310+0000 7f3628c00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 0x7f360806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.311+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f361c0aced0 con 0x7f362c0d2d70 2024-04-26T20:27:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.311+0000 7f362be00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 0x7f360806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.312+0000 7f362be00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 0x7f360806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3614006fd0 tx=0x7f3614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.316+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f362c159a60 con 0x7f362c0d2d70 2024-04-26T20:27:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.520+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f362c0d04d0 con 0x7f362c0d2d70 2024-04-26T20:27:32.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.521+0000 7f3628c00640 1 -- 172.21.15.43:0/4048719398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f361c045360 con 0x7f362c0d2d70 2024-04-26T20:27:32.523 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 msgr2=0x7f360806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 0x7f360806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3614006fd0 tx=0x7f3614008040 comp rx=0 tx=0).stop 2024-04-26T20:27:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 msgr2=0x7f362c159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f361c000f80 tx=0x7f361c004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 shutdown_connections 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f360806c8f0 0x7f360806edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f362c0d3780 0x7f362c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f362c0d31d0 0x7f362c0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 --2- 172.21.15.43:0/4048719398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0d2d70 0x7f362c159000 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 >> 172.21.15.43:0/4048719398 conn(0x7f362c08ec50 msgr2=0x7f362c0d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 shutdown_connections 2024-04-26T20:27:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.524+0000 7f3633a00640 1 -- 172.21.15.43:0/4048719398 wait complete. 2024-04-26T20:27:32.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:27:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.743+0000 7f0201000640 1 Processor -- start 2024-04-26T20:27:32.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.744+0000 7f0201000640 1 -- start start 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.745+0000 7f0201000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 0x7f01fc0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.745+0000 7f0201000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01fc0af8a0 0x7f01fc0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.745+0000 7f0201000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01fc15d2e0 con 0x7f01fc1572b0 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.745+0000 7f0201000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f01fc15d5c0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.745+0000 7f0201000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f01fc15d8a0 con 0x7f01fc0d12e0 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01fc0af8a0 0x7f01fc0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fb400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f01fc1572b0 0x7f01fc0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36700/0 (socket says 172.21.15.43:36700) 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fb400640 1 -- 172.21.15.43:0/2829765256 learned_addr learned my addr 172.21.15.43:0/2829765256 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01fc0af8a0 0x7f01fc0afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32976/0 (socket says 172.21.15.43:32976) 2024-04-26T20:27:32.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01f3e00640 1 --2- 172.21.15.43:0/2829765256 >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 0x7f01fc0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 -- 172.21.15.43:0/2829765256 >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 msgr2=0x7f01fc0b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 --2- 172.21.15.43:0/2829765256 >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 0x7f01fc0b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 -- 172.21.15.43:0/2829765256 >> v1:172.21.15.43:6789/0 conn(0x7f01fc1572b0 legacy=0x7f01fc0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01fbe00640 1 -- 172.21.15.43:0/2829765256 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc15db80 con 0x7f01fc0af8a0 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01faa00640 1 -- 172.21.15.43:0/2829765256 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565305888 0 0) 0x7f01fc15d2e0 con 0x7f01fc1572b0 2024-04-26T20:27:32.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.746+0000 7f01f3e00640 1 --2- 172.21.15.43:0/2829765256 >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 0x7f01fc0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:32.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.747+0000 7f01fbe00640 1 --2- 172.21.15.43:0/2829765256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0afd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0d17e0 tx=0x7f01f402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd9751afdcb06d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f01faa00640 1 -- 172.21.15.43:0/2829765256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4007740 con 0x7f01fc0af8a0 2024-04-26T20:27:32.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f01faa00640 1 -- 172.21.15.43:0/2829765256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40078e0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f01faa00640 1 -- 172.21.15.43:0/2829765256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4007cb0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f0201000640 1 -- 172.21.15.43:0/2829765256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 msgr2=0x7f01fc0afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f0201000640 1 --2- 172.21.15.43:0/2829765256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0afd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0d17e0 tx=0x7f01f402a5f0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.748+0000 7f0201000640 1 -- 172.21.15.43:0/2829765256 shutdown_connections 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.749+0000 7f0201000640 1 --2- 172.21.15.43:0/2829765256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0afd50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.749+0000 7f0201000640 1 --2- 172.21.15.43:0/2829765256 >> v2:172.21.15.164:3300/0 conn(0x7f01fc0d12e0 0x7f01fc0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.749+0000 7f0201000640 1 -- 172.21.15.43:0/2829765256 >> 172.21.15.43:0/2829765256 conn(0x7f01fc0684e0 msgr2=0x7f01fc0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.749+0000 7f0201000640 1 -- 172.21.15.43:0/2829765256 shutdown_connections 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.749+0000 7f0201000640 1 -- 172.21.15.43:0/2829765256 wait complete. 2024-04-26T20:27:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.750+0000 7f0201000640 1 Processor -- start 2024-04-26T20:27:32.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.750+0000 7f0201000640 1 -- start start 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01fc0d12e0 0x7f01fc1e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc1572b0 0x7f01fc1e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc1609f0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc160870 con 0x7f01fc1572b0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.751+0000 7f0201000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01fc160b70 con 0x7f01fc0d12e0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32992/0 (socket says 172.21.15.43:32992) 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 -- 172.21.15.43:0/3092671661 learned_addr learned my addr 172.21.15.43:0/3092671661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fbe00640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc1572b0 0x7f01fc1e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01f3e00640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01fc0d12e0 0x7f01fc1e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 -- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01fc0d12e0 msgr2=0x7f01fc1e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01fc0d12e0 0x7f01fc1e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 -- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc1572b0 msgr2=0x7f01fc1e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc1572b0 0x7f01fc1e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 -- 172.21.15.43:0/3092671661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0b32e0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.752+0000 7f01fb400640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c460 tx=0x7f01ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.753+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec018070 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.753+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec007500 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.753+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec004a20 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.753+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc1e70c0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.753+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc1e7680 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.754+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc0506c0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.755+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01ec004bc0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.756+0000 7f01f3400640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:32.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.756+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f01ec08b350 con 0x7f01fc0af8a0 2024-04-26T20:27:32.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.756+0000 7f01f3e00640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:32.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.757+0000 7f01f3e00640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01fc154a70 tx=0x7f01e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:32.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.761+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01ec0592b0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.964+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f01fc06cb40 con 0x7f01fc0af8a0 2024-04-26T20:27:32.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.965+0000 7f01f3400640 1 -- 172.21.15.43:0/3092671661 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f01ec058bb0 con 0x7f01fc0af8a0 2024-04-26T20:27:32.966 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 msgr2=0x7f01d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01fc154a70 tx=0x7f01e4008040 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 msgr2=0x7f01fc0b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c460 tx=0x7f01ec00c920 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 shutdown_connections 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f01d806c8f0 0x7f01d806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01fc0d12e0 0x7f01fc1e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc1572b0 0x7f01fc1e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 --2- 172.21.15.43:0/3092671661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01fc0af8a0 0x7f01fc0b2da0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 >> 172.21.15.43:0/3092671661 conn(0x7f01fc0684e0 msgr2=0x7f01fc074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 shutdown_connections 2024-04-26T20:27:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:32.968+0000 7f0201000640 1 -- 172.21.15.43:0/3092671661 wait complete. 2024-04-26T20:27:32.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:27:33.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.202+0000 7f2862800640 1 Processor -- start 2024-04-26T20:27:33.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.202+0000 7f2862800640 1 -- start start 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2862800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f285c160d10 0x7f285c0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2862800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 0x7f285c0721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2862800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c163a50 con 0x7f285c06db80 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2862800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f285c163800 con 0x7f285c160d10 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2862800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f285c163d30 con 0x7f285c06e720 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2861400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f285c160d10 0x7f285c0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2860a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f285c06db80 0x7f285c06e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36706/0 (socket says 172.21.15.43:36706) 2024-04-26T20:27:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.204+0000 7f2860a00640 1 -- 172.21.15.43:0/565586931 learned_addr learned my addr 172.21.15.43:0/565586931 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861400640 1 -- 172.21.15.43:0/565586931 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 msgr2=0x7f285c0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861e00640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 0x7f285c0721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f285be00640 1 -- 172.21.15.43:0/565586931 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102128625 0 0) 0x7f285c163a50 con 0x7f285c06db80 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861400640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 0x7f285c0721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861400640 1 -- 172.21.15.43:0/565586931 >> v1:172.21.15.43:6789/0 conn(0x7f285c06db80 legacy=0x7f285c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861400640 1 -- 172.21.15.43:0/565586931 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c164010 con 0x7f285c160d10 2024-04-26T20:27:33.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.205+0000 7f2861e00640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 0x7f285c0721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:33.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.206+0000 7f2861400640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0b5600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f28500097a0 tx=0x7f285002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a5dbf56b808e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.206+0000 7f285be00640 1 -- 172.21.15.43:0/565586931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850037070 con 0x7f285c160d10 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.206+0000 7f285be00640 1 -- 172.21.15.43:0/565586931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2850032440 con 0x7f285c160d10 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.206+0000 7f285be00640 1 -- 172.21.15.43:0/565586931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850032810 con 0x7f285c160d10 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 -- 172.21.15.43:0/565586931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 msgr2=0x7f285c0b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0b5600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f28500097a0 tx=0x7f285002a180 comp rx=0 tx=0).stop 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 -- 172.21.15.43:0/565586931 shutdown_connections 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06e720 0x7f285c0721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 --2- 172.21.15.43:0/565586931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0b5600 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 -- 172.21.15.43:0/565586931 >> 172.21.15.43:0/565586931 conn(0x7f285c068090 msgr2=0x7f285c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 -- 172.21.15.43:0/565586931 shutdown_connections 2024-04-26T20:27:33.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.207+0000 7f2862800640 1 -- 172.21.15.43:0/565586931 wait complete. 2024-04-26T20:27:33.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.210+0000 7f2862800640 1 Processor -- start 2024-04-26T20:27:33.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.210+0000 7f2862800640 1 -- start start 2024-04-26T20:27:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06db80 0x7f285c0d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285c06e720 0x7f285c0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f285c164950 con 0x7f285c160d10 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1610f0 con 0x7f285c06e720 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.211+0000 7f2862800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f285c164ad0 con 0x7f285c06db80 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33000/0 (socket says 172.21.15.43:33000) 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 -- 172.21.15.43:0/1393889803 learned_addr learned my addr 172.21.15.43:0/1393889803 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 -- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06db80 msgr2=0x7f285c0d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06db80 0x7f285c0d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 -- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285c06e720 msgr2=0x7f285c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285c06e720 0x7f285c0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 -- 172.21.15.43:0/1393889803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0ce020 con 0x7f285c160d10 2024-04-26T20:27:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.212+0000 7f2861e00640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f285400cef0 tx=0x7f285400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.213+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285401a070 con 0x7f285c160d10 2024-04-26T20:27:33.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.213+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285400d860 con 0x7f285c160d10 2024-04-26T20:27:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.213+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400dc50 con 0x7f285c160d10 2024-04-26T20:27:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.213+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c150880 con 0x7f285c160d10 2024-04-26T20:27:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.213+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c150e40 con 0x7f285c160d10 2024-04-26T20:27:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.214+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c0506c0 con 0x7f285c160d10 2024-04-26T20:27:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.215+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f285401d070 con 0x7f285c160d10 2024-04-26T20:27:33.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.216+0000 7f2859600640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.216+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f285408c8f0 con 0x7f285c160d10 2024-04-26T20:27:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.216+0000 7f2861400640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.217+0000 7f2861400640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 0x7f283c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2850034000 tx=0x7f285002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.221+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f285405a910 con 0x7f285c160d10 2024-04-26T20:27:33.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.424+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f285c06cb40 con 0x7f285c160d10 2024-04-26T20:27:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.425+0000 7f2859600640 1 -- 172.21.15.43:0/1393889803 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f285405a210 con 0x7f285c160d10 2024-04-26T20:27:33.426 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 msgr2=0x7f283c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 0x7f283c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2850034000 tx=0x7f285002c000 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 msgr2=0x7f285c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f285400cef0 tx=0x7f285400a710 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 shutdown_connections 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f283c06c8f0 0x7f283c06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f285c160d10 0x7f285c0cdae0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285c06e720 0x7f285c0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 --2- 172.21.15.43:0/1393889803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f285c06db80 0x7f285c0d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 >> 172.21.15.43:0/1393889803 conn(0x7f285c068090 msgr2=0x7f285c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 shutdown_connections 2024-04-26T20:27:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.428+0000 7f2862800640 1 -- 172.21.15.43:0/1393889803 wait complete. 2024-04-26T20:27:33.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:27:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.661+0000 7f31bc000640 1 Processor -- start 2024-04-26T20:27:33.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.661+0000 7f31bc000640 1 -- start start 2024-04-26T20:27:33.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31bc000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b40b3490 0x7f31b406f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31bc000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31b406f560 0x7f31b4071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31bc000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40cf060 con 0x7f31b4145590 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31bc000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f31b40d0100 con 0x7f31b40b3490 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31bc000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f31b406f990 con 0x7f31b406f560 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31b8c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b40b3490 0x7f31b406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31b8c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31b40b3490 0x7f31b406f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33006/0 (socket says 172.21.15.43:33006) 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.663+0000 7f31b8c00640 1 -- 172.21.15.43:0/674378668 learned_addr learned my addr 172.21.15.43:0/674378668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:33.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.664+0000 7f31ba000640 1 --2- 172.21.15.43:0/674378668 >> v2:172.21.15.164:3300/0 conn(0x7f31b406f560 0x7f31b4071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.664+0000 7f31b8c00640 1 -- 172.21.15.43:0/674378668 >> v2:172.21.15.164:3300/0 conn(0x7f31b406f560 msgr2=0x7f31b4071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:33.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.664+0000 7f31b8c00640 1 --2- 172.21.15.43:0/674378668 >> v2:172.21.15.164:3300/0 conn(0x7f31b406f560 0x7f31b4071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.664+0000 7f31b8c00640 1 -- 172.21.15.43:0/674378668 >> v1:172.21.15.164:6789/0 conn(0x7f31b4145590 legacy=0x7f31b40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.664+0000 7f31b8c00640 1 -- 172.21.15.43:0/674378668 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40cf340 con 0x7f31b40b3490 2024-04-26T20:27:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.665+0000 7f31b8c00640 1 --2- 172.21.15.43:0/674378668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b40b3490 0x7f31b406f020 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f31b40d1b70 tx=0x7f31a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ef4997dfd51999 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.665+0000 7f31b3e00640 1 -- 172.21.15.43:0/674378668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0037070 con 0x7f31b40b3490 2024-04-26T20:27:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.665+0000 7f31b3e00640 1 -- 172.21.15.43:0/674378668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0032440 con 0x7f31b40b3490 2024-04-26T20:27:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.665+0000 7f31b3e00640 1 -- 172.21.15.43:0/674378668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0032810 con 0x7f31b40b3490 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 -- 172.21.15.43:0/674378668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b40b3490 msgr2=0x7f31b406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 --2- 172.21.15.43:0/674378668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b40b3490 0x7f31b406f020 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f31b40d1b70 tx=0x7f31a002a270 comp rx=0 tx=0).stop 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 -- 172.21.15.43:0/674378668 shutdown_connections 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 --2- 172.21.15.43:0/674378668 >> v2:172.21.15.164:3300/0 conn(0x7f31b406f560 0x7f31b4071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 --2- 172.21.15.43:0/674378668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b40b3490 0x7f31b406f020 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 -- 172.21.15.43:0/674378668 >> 172.21.15.43:0/674378668 conn(0x7f31b40b2900 msgr2=0x7f31b40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 -- 172.21.15.43:0/674378668 shutdown_connections 2024-04-26T20:27:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.666+0000 7f31bc000640 1 -- 172.21.15.43:0/674378668 wait complete. 2024-04-26T20:27:33.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.668+0000 7f31bc000640 1 Processor -- start 2024-04-26T20:27:33.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.668+0000 7f31bc000640 1 -- start start 2024-04-26T20:27:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 0x7f31b41ea840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b4145590 0x7f31b41ead80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40cfbe0 con 0x7f31b406f560 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40cf900 con 0x7f31b40b3490 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.669+0000 7f31bc000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f31b4145970 con 0x7f31b4145590 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33014/0 (socket says 172.21.15.43:33014) 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 -- 172.21.15.43:0/294469051 learned_addr learned my addr 172.21.15.43:0/294469051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b8c00640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 0x7f31b41ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31ba000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b4145590 0x7f31b41ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 -- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b4145590 msgr2=0x7f31b41ead80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b4145590 0x7f31b41ead80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 -- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 msgr2=0x7f31b41ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 0x7f31b41ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b9600640 1 -- 172.21.15.43:0/294469051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41f8640 con 0x7f31b406f560 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.670+0000 7f31b8c00640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 0x7f31b41ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31b9600640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f31a800b3f0 tx=0x7f31a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8015020 con 0x7f31b406f560 2024-04-26T20:27:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a8011830 con 0x7f31b406f560 2024-04-26T20:27:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8011c20 con 0x7f31b406f560 2024-04-26T20:27:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b41f8920 con 0x7f31b406f560 2024-04-26T20:27:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.671+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b41f8ee0 con 0x7f31b406f560 2024-04-26T20:27:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.672+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b4069350 con 0x7f31b406f560 2024-04-26T20:27:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.673+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31a8018020 con 0x7f31b406f560 2024-04-26T20:27:33.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.674+0000 7f31b1600640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:33.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.674+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f31a808bcc0 con 0x7f31b406f560 2024-04-26T20:27:33.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.675+0000 7f31b8c00640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:33.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.675+0000 7f31b8c00640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 0x7f319406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31a0034000 tx=0x7f31a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:33.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.679+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31a8092080 con 0x7f31b406f560 2024-04-26T20:27:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.922+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31b40b1a30 con 0x7f31b406f560 2024-04-26T20:27:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.923+0000 7f31b1600640 1 -- 172.21.15.43:0/294469051 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f31a801f030 con 0x7f31b406f560 2024-04-26T20:27:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:33.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:27:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:27:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 msgr2=0x7f319406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 0x7f319406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31a0034000 tx=0x7f31a002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 msgr2=0x7f31b4140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f31a800b3f0 tx=0x7f31a800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 shutdown_connections 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f31b4145590 0x7f31b41ead80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31b40b3490 0x7f31b41ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 --2- 172.21.15.43:0/294469051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31b406f560 0x7f31b4140180 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 >> 172.21.15.43:0/294469051 conn(0x7f31b40b2900 msgr2=0x7f31b40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.926+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 shutdown_connections 2024-04-26T20:27:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:33.927+0000 7f31bc000640 1 -- 172.21.15.43:0/294469051 wait complete. 2024-04-26T20:27:33.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:34.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.162+0000 7f4763a00640 1 Processor -- start 2024-04-26T20:27:34.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.162+0000 7f4763a00640 1 -- start start 2024-04-26T20:27:34.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f4763a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f475c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f4763a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475c0b0650 con 0x7f475c0d0b40 2024-04-26T20:27:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f4763a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475c163a30 con 0x7f475c15cab0 2024-04-26T20:27:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f4763a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f475c0d0900 con 0x7f475c0d0f80 2024-04-26T20:27:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f475be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f475c0d0b40 0x7f475c15f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36712/0 (socket says 172.21.15.43:36712) 2024-04-26T20:27:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.164+0000 7f475be00640 1 -- 172.21.15.43:0/3562543221 learned_addr learned my addr 172.21.15.43:0/3562543221 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f4761a00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f475c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029538591 0 0) 0x7f475c0b0650 con 0x7f475c0d0b40 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4744003520 con 0x7f475c0d0b40 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976199497 0 0) 0x7f4744003520 con 0x7f475c0d0b40 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 msgr2=0x7f475c15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f475c15fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 >> v1:172.21.15.164:6789/0 conn(0x7f475c15cab0 legacy=0x7f475c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c162c70 con 0x7f475c0d0b40 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f4748001670 con 0x7f475c0d0b40 2024-04-26T20:27:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.165+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 >> v1:172.21.15.43:6789/0 conn(0x7f475c0d0b40 legacy=0x7f475c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47440039e0 0x7f4744003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744004800 0x7f4744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f47440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f474400ba70 con 0x7f4744004800 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f474400bbf0 con 0x7f47440039e0 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f474400bdf0 con 0x7f475c0d0f80 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744004800 0x7f4744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f4761000640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47440039e0 0x7f4744003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f4761a00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f47440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 -- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 msgr2=0x7f47440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c0d0f80 0x7f47440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 -- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47440039e0 msgr2=0x7f4744003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47440039e0 0x7f4744003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.166+0000 7f475be00640 1 -- 172.21.15.43:0/3562543221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4744003520 con 0x7f4744004800 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.168+0000 7f475be00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744004800 0x7f4744004c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f474800c210 tx=0x7f4744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4842b67b097ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.168+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748007d10 con 0x7f4744004800 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.168+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4748005ad0 con 0x7f4744004800 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.168+0000 7f475b400640 1 -- 172.21.15.43:0/3562543221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748007070 con 0x7f4744004800 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 -- 172.21.15.43:0/3562543221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744004800 msgr2=0x7f4744004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 --2- 172.21.15.43:0/3562543221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4744004800 0x7f4744004c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f474800c210 tx=0x7f4744005830 comp rx=0 tx=0).stop 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4761a00640 1 -- 172.21.15.43:0/3562543221 reap_dead start 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 -- 172.21.15.43:0/3562543221 shutdown_connections 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 -- 172.21.15.43:0/3562543221 >> 172.21.15.43:0/3562543221 conn(0x7f475c0b2900 msgr2=0x7f475c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 -- 172.21.15.43:0/3562543221 shutdown_connections 2024-04-26T20:27:34.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.169+0000 7f4763a00640 1 -- 172.21.15.43:0/3562543221 wait complete. 2024-04-26T20:27:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.171+0000 7f4763a00640 1 Processor -- start 2024-04-26T20:27:34.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.171+0000 7f4763a00640 1 -- start start 2024-04-26T20:27:34.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4744004800 0x7f475c14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f475be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f475be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33040/0 (socket says 172.21.15.43:33040) 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c161a10 0x7f475c161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f475c163230 con 0x7f475c14d750 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1635a0 con 0x7f4744004800 2024-04-26T20:27:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f475be00640 1 -- 172.21.15.43:0/3921790090 learned_addr learned my addr 172.21.15.43:0/3921790090 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.172+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f475c0d13f0 con 0x7f475c161a10 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 -- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c161a10 msgr2=0x7f475c161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c161a10 0x7f475c161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 -- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4744004800 msgr2=0x7f475c14edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4744004800 0x7f475c14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c14f600 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f475be00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f474800c330 tx=0x7f475c163510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748040020 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.173+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4748001430 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.174+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748037040 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.174+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c14f8e0 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.174+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c155550 con 0x7f475c14d750 2024-04-26T20:27:34.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.175+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475c069350 con 0x7f475c14d750 2024-04-26T20:27:34.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.176+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4748043020 con 0x7f475c14d750 2024-04-26T20:27:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.177+0000 7f4758c00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 0x7f473c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.177+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f47480ab370 con 0x7f475c14d750 2024-04-26T20:27:34.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.177+0000 7f4761000640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 0x7f473c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.178+0000 7f4761000640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 0x7f473c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4750007fd0 tx=0x7f4750009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.181+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47480792d0 con 0x7f475c14d750 2024-04-26T20:27:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.386+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f475c0b4460 con 0x7f475c14d750 2024-04-26T20:27:34.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.388+0000 7f4758c00640 1 -- 172.21.15.43:0/3921790090 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7f4748078bd0 con 0x7f475c14d750 2024-04-26T20:27:34.390 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:34.390 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 msgr2=0x7f473c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 0x7f473c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4750007fd0 tx=0x7f4750009040 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 msgr2=0x7f475c14cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f474800c330 tx=0x7f475c163510 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 shutdown_connections 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f473c06c8f0 0x7f473c06edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f475c161a10 0x7f475c161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f475c14d750 0x7f475c14cf70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 --2- 172.21.15.43:0/3921790090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4744004800 0x7f475c14edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 >> 172.21.15.43:0/3921790090 conn(0x7f475c0b2900 msgr2=0x7f475c06f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 shutdown_connections 2024-04-26T20:27:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.391+0000 7f4763a00640 1 -- 172.21.15.43:0/3921790090 wait complete. 2024-04-26T20:27:34.413 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:27:34.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:27:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.624+0000 7f06dea00640 1 Processor -- start 2024-04-26T20:27:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.624+0000 7f06dea00640 1 -- start start 2024-04-26T20:27:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.626+0000 7f06dea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06d80b71a0 0x7f06d80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.626+0000 7f06dea00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f06d80b7580 0x7f06d815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06dea00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d80d3a10 con 0x7f06d815ef00 2024-04-26T20:27:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06dea00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f06d80c8830 con 0x7f06d80b71a0 2024-04-26T20:27:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06dea00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f06d80d3cb0 con 0x7f06d80b7580 2024-04-26T20:27:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06dca00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f06d815ef00 0x7f06d8162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36728/0 (socket says 172.21.15.43:36728) 2024-04-26T20:27:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06dca00640 1 -- 172.21.15.43:0/3877150740 learned_addr learned my addr 172.21.15.43:0/3877150740 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3877150740 >> v2:172.21.15.43:3300/0 conn(0x7f06d80b71a0 0x7f06d80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.627+0000 7f06d7400640 1 --2- 172.21.15.43:0/3877150740 >> v2:172.21.15.164:3300/0 conn(0x7f06d80b7580 0x7f06d815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.628+0000 7f06d7e00640 1 -- 172.21.15.43:0/3877150740 >> v2:172.21.15.164:3300/0 conn(0x7f06d80b7580 msgr2=0x7f06d815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:34.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.628+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3877150740 >> v2:172.21.15.164:3300/0 conn(0x7f06d80b7580 0x7f06d815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.628+0000 7f06d7e00640 1 -- 172.21.15.43:0/3877150740 >> v1:172.21.15.43:6789/0 conn(0x7f06d815ef00 legacy=0x7f06d8162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.628+0000 7f06d7e00640 1 -- 172.21.15.43:0/3877150740 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d80c8af0 con 0x7f06d80b71a0 2024-04-26T20:27:34.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.629+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3877150740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d80b71a0 0x7f06d80d34d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f06d808d0b0 tx=0x7f06c802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca0635ab9355cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.630+0000 7f06d6a00640 1 -- 172.21.15.43:0/3877150740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8037070 con 0x7f06d80b71a0 2024-04-26T20:27:34.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.630+0000 7f06d6a00640 1 -- 172.21.15.43:0/3877150740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c8032440 con 0x7f06d80b71a0 2024-04-26T20:27:34.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.630+0000 7f06d6a00640 1 -- 172.21.15.43:0/3877150740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c802ae60 con 0x7f06d80b71a0 2024-04-26T20:27:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 -- 172.21.15.43:0/3877150740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d80b71a0 msgr2=0x7f06d80d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 --2- 172.21.15.43:0/3877150740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d80b71a0 0x7f06d80d34d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f06d808d0b0 tx=0x7f06c802a110 comp rx=0 tx=0).stop 2024-04-26T20:27:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 -- 172.21.15.43:0/3877150740 shutdown_connections 2024-04-26T20:27:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 --2- 172.21.15.43:0/3877150740 >> v2:172.21.15.164:3300/0 conn(0x7f06d80b7580 0x7f06d815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 --2- 172.21.15.43:0/3877150740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d80b71a0 0x7f06d80d34d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 -- 172.21.15.43:0/3877150740 >> 172.21.15.43:0/3877150740 conn(0x7f06d80d4d70 msgr2=0x7f06d80d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 -- 172.21.15.43:0/3877150740 shutdown_connections 2024-04-26T20:27:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.631+0000 7f06dea00640 1 -- 172.21.15.43:0/3877150740 wait complete. 2024-04-26T20:27:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.633+0000 7f06dea00640 1 Processor -- start 2024-04-26T20:27:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.633+0000 7f06dea00640 1 -- start start 2024-04-26T20:27:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06d80b71a0 0x7f06d80cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06d80b7580 0x7f06d8161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06d815f330 con 0x7f06d815ef00 2024-04-26T20:27:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80c90b0 con 0x7f06d80b7580 2024-04-26T20:27:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dea00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f06d815f4b0 con 0x7f06d80b71a0 2024-04-26T20:27:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33064/0 (socket says 172.21.15.43:33064) 2024-04-26T20:27:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dca00640 1 -- 172.21.15.43:0/3880877147 learned_addr learned my addr 172.21.15.43:0/3880877147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06d80b71a0 0x7f06d80cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.634+0000 7f06dca00640 1 -- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06d80b71a0 msgr2=0x7f06d80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06d7400640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06d80b7580 0x7f06d8161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dca00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06d80b71a0 0x7f06d80cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dca00640 1 -- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06d80b7580 msgr2=0x7f06d8161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dca00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06d80b7580 0x7f06d8161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dca00640 1 -- 172.21.15.43:0/3880877147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d815f910 con 0x7f06d815ef00 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dca00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f06d000cab0 tx=0x7f06d000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0017070 con 0x7f06d815ef00 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d000ac30 con 0x7f06d815ef00 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0012400 con 0x7f06d815ef00 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d815fbb0 con 0x7f06d815ef00 2024-04-26T20:27:34.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.635+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d8160170 con 0x7f06d815ef00 2024-04-26T20:27:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.637+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d8069350 con 0x7f06d815ef00 2024-04-26T20:27:34.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.638+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06d001a070 con 0x7f06d815ef00 2024-04-26T20:27:34.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.639+0000 7f06cfe00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 0x7f06b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:34.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.639+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f06d008bb50 con 0x7f06d815ef00 2024-04-26T20:27:34.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.639+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 0x7f06b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.640+0000 7f06d7e00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 0x7f06b806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06c8034000 tx=0x7f06c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:34.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.644+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06d0059500 con 0x7f06d815ef00 2024-04-26T20:27:34.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.892+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d8074fe0 con 0x7f06d815ef00 2024-04-26T20:27:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.893+0000 7f06cfe00640 1 -- 172.21.15.43:0/3880877147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f06d005d800 con 0x7f06d815ef00 2024-04-26T20:27:34.895 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:34.895 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:27:34.896 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 msgr2=0x7f06b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 0x7f06b806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06c8034000 tx=0x7f06c802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 msgr2=0x7f06d8161540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f06d000cab0 tx=0x7f06d000cf70 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 shutdown_connections 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f06b806c8f0 0x7f06b806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d815ef00 0x7f06d8161540 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06d80b7580 0x7f06d8161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 --2- 172.21.15.43:0/3880877147 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f06d80b71a0 0x7f06d80cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:34.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 >> 172.21.15.43:0/3880877147 conn(0x7f06d80d4d70 msgr2=0x7f06d806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 shutdown_connections 2024-04-26T20:27:34.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:34.896+0000 7f06dea00640 1 -- 172.21.15.43:0/3880877147 wait complete. 2024-04-26T20:27:34.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:27:35.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.119+0000 7f120a200640 1 Processor -- start 2024-04-26T20:27:35.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.119+0000 7f120a200640 1 -- start start 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.121+0000 7f120a200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12040cd180 0x7f1204072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.121+0000 7f120a200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 0x7f12040cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.121+0000 7f120a200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12040cd560 con 0x7f12040ce2b0 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.121+0000 7f120a200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1204072d00 con 0x7f12040cd180 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.121+0000 7f120a200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1204072fe0 con 0x7f120415dc50 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1208e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12040cd180 0x7f1204072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1209800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f12040ce2b0 0x7f12041607d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36730/0 (socket says 172.21.15.43:36730) 2024-04-26T20:27:35.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1209800640 1 -- 172.21.15.43:0/1885262694 learned_addr learned my addr 172.21.15.43:0/1885262694 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1203e00640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 0x7f12040cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1208e00640 1 -- 172.21.15.43:0/1885262694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 msgr2=0x7f12040cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1208e00640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 0x7f12040cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1208e00640 1 -- 172.21.15.43:0/1885262694 >> v1:172.21.15.43:6789/0 conn(0x7f12040ce2b0 legacy=0x7f12041607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1208e00640 1 -- 172.21.15.43:0/1885262694 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12040732c0 con 0x7f12040cd180 2024-04-26T20:27:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1203e00640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 0x7f12040cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.122+0000 7f1203400640 1 -- 172.21.15.43:0/1885262694 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724639513 0 0) 0x7f12040cd560 con 0x7f12040ce2b0 2024-04-26T20:27:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.123+0000 7f1208e00640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12040cd180 0x7f1204072620 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11f80097a0 tx=0x7f11f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2be33e05f8c66b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f1203400640 1 -- 172.21.15.43:0/1885262694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8037070 con 0x7f12040cd180 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f1203400640 1 -- 172.21.15.43:0/1885262694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8032440 con 0x7f12040cd180 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f1203400640 1 -- 172.21.15.43:0/1885262694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8032810 con 0x7f12040cd180 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 -- 172.21.15.43:0/1885262694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12040cd180 msgr2=0x7f1204072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12040cd180 0x7f1204072620 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11f80097a0 tx=0x7f11f802a180 comp rx=0 tx=0).stop 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 -- 172.21.15.43:0/1885262694 shutdown_connections 2024-04-26T20:27:35.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f120415dc50 0x7f12040cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 --2- 172.21.15.43:0/1885262694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12040cd180 0x7f1204072620 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.124+0000 7f120a200640 1 -- 172.21.15.43:0/1885262694 >> 172.21.15.43:0/1885262694 conn(0x7f1204068090 msgr2=0x7f1204068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.125+0000 7f120a200640 1 -- 172.21.15.43:0/1885262694 shutdown_connections 2024-04-26T20:27:35.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.125+0000 7f120a200640 1 -- 172.21.15.43:0/1885262694 wait complete. 2024-04-26T20:27:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.126+0000 7f120a200640 1 Processor -- start 2024-04-26T20:27:35.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.126+0000 7f120a200640 1 -- start start 2024-04-26T20:27:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12040cd180 0x7f12040ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12040ce2b0 0x7f120413dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1204073da0 con 0x7f120415dc50 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12040ce700 con 0x7f12040ce2b0 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.127+0000 7f120a200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f120406ead0 con 0x7f12040cd180 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33076/0 (socket says 172.21.15.43:33076) 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 -- 172.21.15.43:0/2218155365 learned_addr learned my addr 172.21.15.43:0/2218155365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1208e00640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12040cd180 0x7f12040ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1203e00640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12040ce2b0 0x7f120413dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 -- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12040cd180 msgr2=0x7f12040ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12040cd180 0x7f12040ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 -- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12040ce2b0 msgr2=0x7f120413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12040ce2b0 0x7f120413dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 -- 172.21.15.43:0/2218155365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12040aa4b0 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.128+0000 7f1209800640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c7d0 tx=0x7f11fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.129+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc018070 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.129+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc00a640 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.129+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00aa70 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.129+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12040aa790 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.129+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12040aad50 con 0x7f120415dc50 2024-04-26T20:27:35.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.130+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1204069350 con 0x7f120415dc50 2024-04-26T20:27:35.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.131+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11fc01b070 con 0x7f120415dc50 2024-04-26T20:27:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.132+0000 7f1200c00640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 0x7f11e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.132+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f11fc08b680 con 0x7f120415dc50 2024-04-26T20:27:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.133+0000 7f1208e00640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 0x7f11e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.133+0000 7f1208e00640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 0x7f11e406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f11f8034000 tx=0x7f11f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.137+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11fc0595e0 con 0x7f120415dc50 2024-04-26T20:27:35.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.382+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12040d0670 con 0x7f120415dc50 2024-04-26T20:27:35.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.383+0000 7f1200c00640 1 -- 172.21.15.43:0/2218155365 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f11fc058ee0 con 0x7f120415dc50 2024-04-26T20:27:35.385 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:35.385 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1013666487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1013666487},{"type":"v1","addr":"172.21.15.164:6834","nonce":1013666487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3545081495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":3545081495},{"type":"v1","addr":"172.21.15.43:6837","nonce":3545081495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/3625692972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3625692972},{"type":"v1","addr":"172.21.15.43:6836","nonce":3625692972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:25:57.479955+0000","modified":"2024-04-26T20:26:24.666310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6835/2023737548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":2023737548},{"type":"v1","addr":"172.21.15.164:6835","nonce":2023737548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4273,"qdb_cluster":[4273]},"id":1}]} 2024-04-26T20:27:35.385 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-04-26T20:27:35.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 msgr2=0x7f11e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 0x7f11e406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f11f8034000 tx=0x7f11f802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:35.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 msgr2=0x7f120415c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c7d0 tx=0x7f11fc00cc90 comp rx=0 tx=0).stop 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 shutdown_connections 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f11e406c8f0 0x7f11e406edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f120415dc50 0x7f120415c2d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12040ce2b0 0x7f120413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 --2- 172.21.15.43:0/2218155365 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12040cd180 0x7f12040ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 >> 172.21.15.43:0/2218155365 conn(0x7f1204068090 msgr2=0x7f120406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.386+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 shutdown_connections 2024-04-26T20:27:35.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.387+0000 7f120a200640 1 -- 172.21.15.43:0/2218155365 wait complete. 2024-04-26T20:27:35.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:35.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.617+0000 7fe0e5600640 1 Processor -- start 2024-04-26T20:27:35.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.618+0000 7fe0e5600640 1 -- start start 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.619+0000 7fe0e5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0e00d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.619+0000 7fe0e5600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e0146ba0 con 0x7fe0e01467c0 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.619+0000 7fe0e5600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e00731c0 con 0x7fe0e00d1060 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.619+0000 7fe0e5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e0073dd0 con 0x7fe0e0072b70 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.620+0000 7fe0dea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0e01467c0 0x7fe0e00b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36738/0 (socket says 172.21.15.43:36738) 2024-04-26T20:27:35.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.620+0000 7fe0dea00640 1 -- 172.21.15.43:0/3727839773 learned_addr learned my addr 172.21.15.43:0/3727839773 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.620+0000 7fe0de000640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0e00d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.620+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853173555 0 0) 0x7fe0e0146ba0 con 0x7fe0e01467c0 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.620+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0c4003520 con 0x7fe0e01467c0 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3020916110 0 0) 0x7fe0c4003520 con 0x7fe0e01467c0 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 msgr2=0x7fe0e00d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0e00d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 >> v1:172.21.15.164:6789/0 conn(0x7fe0e00d1060 legacy=0x7fe0e006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e00734a0 con 0x7fe0e01467c0 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fe0d00019f0 con 0x7fe0e01467c0 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 >> v1:172.21.15.43:6789/0 conn(0x7fe0e01467c0 legacy=0x7fe0e00b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004d40 0x7fe0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0073dd0 con 0x7fe0c4003f60 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c400baa0 con 0x7fe0c4004d40 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c400bca0 con 0x7fe0e0072b70 2024-04-26T20:27:35.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.621+0000 7fe0de000640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0dea00640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0de000640 1 -- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 msgr2=0x7fe0c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0de000640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e0072b70 0x7fe0c40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0de000640 1 -- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004d40 msgr2=0x7fe0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0de000640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0c4004d40 0x7fe0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.622+0000 7fe0de000640 1 -- 172.21.15.43:0/3727839773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4003520 con 0x7fe0c4003f60 2024-04-26T20:27:35.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.623+0000 7fe0de000640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0c4004800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009ac0 tx=0x7fe0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca278f35adbb25d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.623+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc036070 con 0x7fe0c4003f60 2024-04-26T20:27:35.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.623+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc031400 con 0x7fe0c4003f60 2024-04-26T20:27:35.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.623+0000 7fe0dd600640 1 -- 172.21.15.43:0/3727839773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc009690 con 0x7fe0c4003f60 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 -- 172.21.15.43:0/3727839773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 msgr2=0x7fe0c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 --2- 172.21.15.43:0/3727839773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0c4004800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009ac0 tx=0x7fe0c4005830 comp rx=0 tx=0).stop 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0df400640 1 -- 172.21.15.43:0/3727839773 reap_dead start 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 -- 172.21.15.43:0/3727839773 shutdown_connections 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 -- 172.21.15.43:0/3727839773 >> 172.21.15.43:0/3727839773 conn(0x7fe0e00b2900 msgr2=0x7fe0e00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 -- 172.21.15.43:0/3727839773 shutdown_connections 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.624+0000 7fe0e5600640 1 -- 172.21.15.43:0/3727839773 wait complete. 2024-04-26T20:27:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.626+0000 7fe0e5600640 1 Processor -- start 2024-04-26T20:27:35.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.626+0000 7fe0e5600640 1 -- start start 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e01e8280 0x7fe0e01f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0e01f2950 0x7fe0e0141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00d0b20 con 0x7fe0c4003f60 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e0073a60 con 0x7fe0e01f2950 2024-04-26T20:27:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0e5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00d0ca0 con 0x7fe0e01e8280 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0dea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33106/0 (socket says 172.21.15.43:33106) 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0dea00640 1 -- 172.21.15.43:0/934251373 learned_addr learned my addr 172.21.15.43:0/934251373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.627+0000 7fe0df400640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0e01f2950 0x7fe0e0141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 -- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e01e8280 msgr2=0x7fe0e01f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e01e8280 0x7fe0e01f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 -- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0e01f2950 msgr2=0x7fe0e0141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0e01f2950 0x7fe0e0141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 -- 172.21.15.43:0/934251373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e0142820 con 0x7fe0c4003f60 2024-04-26T20:27:35.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.628+0000 7fe0dea00640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0002210 tx=0x7fe0d000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.630+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0019070 con 0x7fe0c4003f60 2024-04-26T20:27:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.630+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0004d60 con 0x7fe0c4003f60 2024-04-26T20:27:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.630+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00072d0 con 0x7fe0c4003f60 2024-04-26T20:27:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.630+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e01e5610 con 0x7fe0c4003f60 2024-04-26T20:27:35.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.631+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e01f2db0 con 0x7fe0c4003f60 2024-04-26T20:27:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.632+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e01e86c0 con 0x7fe0c4003f60 2024-04-26T20:27:35.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.633+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0d001c070 con 0x7fe0c4003f60 2024-04-26T20:27:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.634+0000 7fe0d6a00640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 0x7fe0c0073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.634+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7fe0d008c080 con 0x7fe0c4003f60 2024-04-26T20:27:35.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.634+0000 7fe0de000640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 0x7fe0c0073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:35.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.635+0000 7fe0de000640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 0x7fe0c0073240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:35.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.638+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0e01e86c0 con 0x7fe0c4003f60 2024-04-26T20:27:35.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.841+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0e0069350 con 0x7fe0c4003f60 2024-04-26T20:27:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.844+0000 7fe0d6a00640 1 -- 172.21.15.43:0/934251373 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18884 (secure 0 0 0) 0x7fe0d0059aa0 con 0x7fe0c4003f60 2024-04-26T20:27:35.846 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:35.846 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":26,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:26:44.307703+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:25:54.942670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:25:56.403597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:26:00.033245+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.14","mappings":[{"from":4,"to":2},{"from":7,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 msgr2=0x7fe0c0073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 0x7fe0c0073240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc033000 tx=0x7fe0cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 msgr2=0x7fe0e01e7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0002210 tx=0x7fe0d000b560 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 shutdown_connections 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe0c0070d80 0x7fe0c0073240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe0e01e8280 0x7fe0e01f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0e01f2950 0x7fe0e0141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 --2- 172.21.15.43:0/934251373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0c4003f60 0x7fe0e01e7d40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:35.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.846+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 >> 172.21.15.43:0/934251373 conn(0x7fe0e00b2900 msgr2=0x7fe0e006f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.847+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 shutdown_connections 2024-04-26T20:27:35.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:35.847+0000 7fe0e5600640 1 -- 172.21.15.43:0/934251373 wait complete. 2024-04-26T20:27:35.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:27:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.061+0000 7f7bba800640 1 Processor -- start 2024-04-26T20:27:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.061+0000 7f7bba800640 1 -- start start 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.062+0000 7f7bba800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7bb40af710 0x7f7bb406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.062+0000 7f7bba800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb4167ae0 con 0x7f7bb40afc60 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bba800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb406dcd0 con 0x7f7bb40726c0 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bba800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb406dfb0 con 0x7f7bb40af710 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bb8a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7bb40afc60 0x7f7bb406e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58818/0 (socket says 172.21.15.43:58818) 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bb9e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7bb40af710 0x7f7bb406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bb8a00640 1 -- 172.21.15.43:0/981842230 learned_addr learned my addr 172.21.15.43:0/981842230 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.063+0000 7f7bb3e00640 1 -- 172.21.15.43:0/981842230 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513077244 0 0) 0x7f7bb4167ae0 con 0x7f7bb40afc60 2024-04-26T20:27:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.064+0000 7f7bb3e00640 1 -- 172.21.15.43:0/981842230 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b9c003520 con 0x7f7bb40afc60 2024-04-26T20:27:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.064+0000 7f7bb9e00640 1 -- 172.21.15.43:0/981842230 >> v1:172.21.15.164:6789/0 conn(0x7f7bb40726c0 legacy=0x7f7bb4072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.064+0000 7f7bb9e00640 1 -- 172.21.15.43:0/981842230 >> v1:172.21.15.43:6789/0 conn(0x7f7bb40afc60 legacy=0x7f7bb406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.064+0000 7f7bb9e00640 1 -- 172.21.15.43:0/981842230 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb406e290 con 0x7f7bb40af710 2024-04-26T20:27:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.065+0000 7f7bb9e00640 1 --2- 172.21.15.43:0/981842230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40af710 0x7f7bb406eec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0097a0 tx=0x7f7bac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb8ae2d4285bdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.065+0000 7f7bb3e00640 1 -- 172.21.15.43:0/981842230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac037070 con 0x7f7bb40af710 2024-04-26T20:27:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.065+0000 7f7bb3e00640 1 -- 172.21.15.43:0/981842230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bac032440 con 0x7f7bb40af710 2024-04-26T20:27:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.065+0000 7f7bb3e00640 1 -- 172.21.15.43:0/981842230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0327f0 con 0x7f7bb40af710 2024-04-26T20:27:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 -- 172.21.15.43:0/981842230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40af710 msgr2=0x7f7bb406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 --2- 172.21.15.43:0/981842230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40af710 0x7f7bb406eec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0097a0 tx=0x7f7bac02a230 comp rx=0 tx=0).stop 2024-04-26T20:27:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 -- 172.21.15.43:0/981842230 shutdown_connections 2024-04-26T20:27:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 --2- 172.21.15.43:0/981842230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40af710 0x7f7bb406eec0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 -- 172.21.15.43:0/981842230 >> 172.21.15.43:0/981842230 conn(0x7f7bb4068090 msgr2=0x7f7bb4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 -- 172.21.15.43:0/981842230 shutdown_connections 2024-04-26T20:27:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.066+0000 7f7bba800640 1 -- 172.21.15.43:0/981842230 wait complete. 2024-04-26T20:27:36.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.067+0000 7f7bba800640 1 Processor -- start 2024-04-26T20:27:36.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.068+0000 7f7bba800640 1 -- start start 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.068+0000 7f7bba800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bba800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb40af710 0x7f7bb4161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bba800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bb40afc60 0x7f7bb40ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bba800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41635a0 con 0x7f7bb40726c0 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bba800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4163420 con 0x7f7bb40af710 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bba800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb40cdaf0 con 0x7f7bb40afc60 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52060/0 (socket says 172.21.15.43:52060) 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 -- 172.21.15.43:0/594904420 learned_addr learned my addr 172.21.15.43:0/594904420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 -- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bb40afc60 msgr2=0x7f7bb40ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bb40afc60 0x7f7bb40ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 -- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb40af710 msgr2=0x7f7bb4161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb8a00640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb40af710 0x7f7bb4161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb40af710 0x7f7bb4161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.069+0000 7f7bb9400640 1 -- 172.21.15.43:0/594904420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb4162130 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bb9400640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c730 tx=0x7f7ba800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8003b70 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba8003d30 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800aa90 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb4162410 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.070+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41629d0 con 0x7f7bb40726c0 2024-04-26T20:27:36.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.071+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb41674d0 con 0x7f7bb40726c0 2024-04-26T20:27:36.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.072+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ba800ac30 con 0x7f7bb40726c0 2024-04-26T20:27:36.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.074+0000 7f7bb1600640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 0x7f7b9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.074+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6772+0+0 (secure 0 0 0) 0x7f7ba808be70 con 0x7f7bb40726c0 2024-04-26T20:27:36.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.074+0000 7f7bb8a00640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 0x7f7b9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.075+0000 7f7bb8a00640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 0x7f7b9406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40b30a0 tx=0x7f7ba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.078+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bb41674d0 con 0x7f7bb40726c0 2024-04-26T20:27:36.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.325+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7bb4069350 con 0x7f7bb40726c0 2024-04-26T20:27:36.327 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-26T20:27:36.326+0000 7f2bfa400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:27:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.331+0000 7f7bb1600640 1 -- 172.21.15.43:0/594904420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f7ba8059e60 con 0x7f7bb40726c0 2024-04-26T20:27:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 msgr2=0x7f7b9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 0x7f7b9406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40b30a0 tx=0x7f7ba0009040 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 msgr2=0x7f7bb41645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c730 tx=0x7f7ba800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 shutdown_connections 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7b9406c920 0x7f7b9406ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bb40afc60 0x7f7bb40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bb40af710 0x7f7bb4161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 --2- 172.21.15.43:0/594904420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bb40726c0 0x7f7bb41645e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 >> 172.21.15.43:0/594904420 conn(0x7f7bb4068090 msgr2=0x7f7bb406f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 shutdown_connections 2024-04-26T20:27:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.334+0000 7f7bba800640 1 -- 172.21.15.43:0/594904420 wait complete. 2024-04-26T20:27:36.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:27:36.369 INFO:tasks.ceph.mds.b.smithi164.stderr: -6130> 2024-04-26T20:26:00.843+0000 7f65d1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:27:36.455 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-04-26T20:27:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.563+0000 7f8fc8e00640 1 Processor -- start 2024-04-26T20:27:36.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.564+0000 7f8fc8e00640 1 -- start start 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.565+0000 7f8fc8e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8fc4155cd0 0x7f8fc4072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.565+0000 7f8fc8e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fc40b5b10 0x7f8fc4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.565+0000 7f8fc8e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc415f920 con 0x7f8fc4072b40 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.565+0000 7f8fc8e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc415fc00 con 0x7f8fc40b5b10 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.565+0000 7f8fc8e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc415fee0 con 0x7f8fc4155cd0 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc3e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fc40b5b10 0x7f8fc4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fbbe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8fc4072b40 0x7f8fc40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58830/0 (socket says 172.21.15.43:58830) 2024-04-26T20:27:36.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fbbe00640 1 -- 172.21.15.43:0/3147760372 learned_addr learned my addr 172.21.15.43:0/3147760372 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc3400640 1 --2- 172.21.15.43:0/3147760372 >> v2:172.21.15.164:3300/0 conn(0x7f8fc4155cd0 0x7f8fc4072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc3e00640 1 -- 172.21.15.43:0/3147760372 >> v2:172.21.15.164:3300/0 conn(0x7f8fc4155cd0 msgr2=0x7f8fc4072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc3e00640 1 --2- 172.21.15.43:0/3147760372 >> v2:172.21.15.164:3300/0 conn(0x7f8fc4155cd0 0x7f8fc4072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc3e00640 1 -- 172.21.15.43:0/3147760372 >> v1:172.21.15.43:6789/0 conn(0x7f8fc4072b40 legacy=0x7f8fc40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.567+0000 7f8fc3e00640 1 -- 172.21.15.43:0/3147760372 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41601c0 con 0x7f8fc40b5b10 2024-04-26T20:27:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.566+0000 7f8fc2a00640 1 -- 172.21.15.43:0/3147760372 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437649252 0 0) 0x7f8fc415f920 con 0x7f8fc4072b40 2024-04-26T20:27:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.568+0000 7f8fc3e00640 1 --2- 172.21.15.43:0/3147760372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc4162f00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40d1320 tx=0x7f8fbc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cd61ffd285b08f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.568+0000 7f8fc2a00640 1 -- 172.21.15.43:0/3147760372 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007740 con 0x7f8fc40b5b10 2024-04-26T20:27:36.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.568+0000 7f8fc2a00640 1 -- 172.21.15.43:0/3147760372 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0078e0 con 0x7f8fc40b5b10 2024-04-26T20:27:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.568+0000 7f8fc2a00640 1 -- 172.21.15.43:0/3147760372 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007cb0 con 0x7f8fc40b5b10 2024-04-26T20:27:36.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3147760372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 msgr2=0x7f8fc4162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3147760372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc4162f00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40d1320 tx=0x7f8fbc02a710 comp rx=0 tx=0).stop 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3147760372 shutdown_connections 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3147760372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc4162f00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3147760372 >> v2:172.21.15.164:3300/0 conn(0x7f8fc4155cd0 0x7f8fc4072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3147760372 >> 172.21.15.43:0/3147760372 conn(0x7f8fc4068090 msgr2=0x7f8fc40684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3147760372 shutdown_connections 2024-04-26T20:27:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.569+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3147760372 wait complete. 2024-04-26T20:27:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.570+0000 7f8fc8e00640 1 Processor -- start 2024-04-26T20:27:36.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.571+0000 7f8fc8e00640 1 -- start start 2024-04-26T20:27:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fc4072b40 0x7f8fc4153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 0x7f8fc4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4156230 con 0x7f8fc40b5b10 2024-04-26T20:27:36.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41560b0 con 0x7f8fc4155cd0 2024-04-26T20:27:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc8e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc415f3f0 con 0x7f8fc4072b40 2024-04-26T20:27:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52066/0 (socket says 172.21.15.43:52066) 2024-04-26T20:27:36.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 -- 172.21.15.43:0/3792797797 learned_addr learned my addr 172.21.15.43:0/3792797797 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc3e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 0x7f8fc4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc3400640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fc4072b40 0x7f8fc4153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 -- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fc4072b40 msgr2=0x7f8fc4153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fc4072b40 0x7f8fc4153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 -- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 msgr2=0x7f8fc4159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 0x7f8fc4159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fbbe00640 1 -- 172.21.15.43:0/3792797797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc4159f60 con 0x7f8fc40b5b10 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.572+0000 7f8fc3e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 0x7f8fc4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:36.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fbbe00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c4d0 tx=0x7f8fac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac018070 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac007500 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac004a20 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc415a230 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.573+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc415a7f0 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.574+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc40506c0 con 0x7f8fc40b5b10 2024-04-26T20:27:36.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.575+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fac01b070 con 0x7f8fc40b5b10 2024-04-26T20:27:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.577+0000 7f8fbb400640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.577+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6858+0+0 (secure 0 0 0) 0x7f8fac08b370 con 0x7f8fc40b5b10 2024-04-26T20:27:36.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.577+0000 7f8fc3400640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:36.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.578+0000 7f8fc3400640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 0x7f8fa006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:36.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.581+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8fac059280 con 0x7f8fc40b5b10 2024-04-26T20:27:36.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:36.828+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8fc40b0fc0 con 0x7f8fc40b5b10 2024-04-26T20:27:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.334+0000 7f8fbb400640 1 -- 172.21.15.43:0/3792797797 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f8fac058b80 con 0x7f8fc40b5b10 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.337+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 msgr2=0x7f8fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 0x7f8fa006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb4008040 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 msgr2=0x7f8fc415b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c4d0 tx=0x7f8fac00c990 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 shutdown_connections 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8fa006c8f0 0x7f8fa006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4155cd0 0x7f8fc4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fc40b5b10 0x7f8fc415b2d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 --2- 172.21.15.43:0/3792797797 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fc4072b40 0x7f8fc4153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 >> 172.21.15.43:0/3792797797 conn(0x7f8fc4068090 msgr2=0x7f8fc4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 shutdown_connections 2024-04-26T20:27:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.338+0000 7f8fc8e00640 1 -- 172.21.15.43:0/3792797797 wait complete. 2024-04-26T20:27:37.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:27:37.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.578+0000 7fce85600640 1 Processor -- start 2024-04-26T20:27:37.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.578+0000 7fce85600640 1 -- start start 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce85600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce8006fec0 0x7fce8006ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce85600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 0x7fce80071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce85600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce8006df30 con 0x7fce80146800 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce85600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fce80146be0 con 0x7fce8006fec0 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce85600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fce8006e860 con 0x7fce8006f230 2024-04-26T20:27:37.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce7e000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce8006fec0 0x7fce8006ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce7e000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce8006fec0 0x7fce8006ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52080/0 (socket says 172.21.15.43:52080) 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.580+0000 7fce7e000640 1 -- 172.21.15.43:0/3188208861 learned_addr learned my addr 172.21.15.43:0/3188208861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7f400640 1 --2- 172.21.15.43:0/3188208861 >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 0x7fce80071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7e000640 1 -- 172.21.15.43:0/3188208861 >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 msgr2=0x7fce80071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7e000640 1 --2- 172.21.15.43:0/3188208861 >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 0x7fce80071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7e000640 1 -- 172.21.15.43:0/3188208861 >> v1:172.21.15.164:6789/0 conn(0x7fce80146800 legacy=0x7fce800cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7e000640 1 -- 172.21.15.43:0/3188208861 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8006e210 con 0x7fce8006fec0 2024-04-26T20:27:37.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.581+0000 7fce7f400640 1 --2- 172.21.15.43:0/3188208861 >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 0x7fce80071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:37.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.582+0000 7fce7e000640 1 --2- 172.21.15.43:0/3188208861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006fec0 0x7fce8006ecf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fce80157a90 tx=0x7fce6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a3e6122a1dbaca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:37.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.583+0000 7fce7d600640 1 -- 172.21.15.43:0/3188208861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c037070 con 0x7fce8006fec0 2024-04-26T20:27:37.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.583+0000 7fce7d600640 1 -- 172.21.15.43:0/3188208861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c032440 con 0x7fce8006fec0 2024-04-26T20:27:37.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.583+0000 7fce7d600640 1 -- 172.21.15.43:0/3188208861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c032810 con 0x7fce8006fec0 2024-04-26T20:27:37.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.583+0000 7fce85600640 1 -- 172.21.15.43:0/3188208861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006fec0 msgr2=0x7fce8006ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.583+0000 7fce85600640 1 --2- 172.21.15.43:0/3188208861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006fec0 0x7fce8006ecf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fce80157a90 tx=0x7fce6c02a230 comp rx=0 tx=0).stop 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 -- 172.21.15.43:0/3188208861 shutdown_connections 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 --2- 172.21.15.43:0/3188208861 >> v2:172.21.15.164:3300/0 conn(0x7fce8006f230 0x7fce80071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 --2- 172.21.15.43:0/3188208861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006fec0 0x7fce8006ecf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 -- 172.21.15.43:0/3188208861 >> 172.21.15.43:0/3188208861 conn(0x7fce800b2790 msgr2=0x7fce800b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 -- 172.21.15.43:0/3188208861 shutdown_connections 2024-04-26T20:27:37.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.584+0000 7fce85600640 1 -- 172.21.15.43:0/3188208861 wait complete. 2024-04-26T20:27:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.585+0000 7fce85600640 1 Processor -- start 2024-04-26T20:27:37.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.586+0000 7fce85600640 1 -- start start 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.586+0000 7fce85600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 0x7fce8015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.586+0000 7fce85600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8006fec0 0x7fce8013e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce85600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce85600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce800723a0 con 0x7fce80146800 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce85600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d0520 con 0x7fce8006fec0 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce85600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fce80072520 con 0x7fce8006f230 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52084/0 (socket says 172.21.15.43:52084) 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 -- 172.21.15.43:0/3811271893 learned_addr learned my addr 172.21.15.43:0/3811271893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7e000640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8006fec0 0x7fce8013e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7ea00640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 0x7fce8015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 -- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 msgr2=0x7fce8015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 0x7fce8015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 -- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8006fec0 msgr2=0x7fce8013e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8006fec0 0x7fce8013e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7f400640 1 -- 172.21.15.43:0/3811271893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8013eeb0 con 0x7fce80146800 2024-04-26T20:27:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.587+0000 7fce7ea00640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 0x7fce8015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce7f400640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fce7800cca0 tx=0x7fce7800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7800f680 con 0x7fce80146800 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7800f840 con 0x7fce80146800 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7800fc30 con 0x7fce80146800 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8013f190 con 0x7fce80146800 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.588+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8013f750 con 0x7fce80146800 2024-04-26T20:27:37.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.589+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80069350 con 0x7fce80146800 2024-04-26T20:27:37.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.591+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce7800e3e0 con 0x7fce80146800 2024-04-26T20:27:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.591+0000 7fce76a00640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 0x7fce6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.592+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6858+0+0 (secure 0 0 0) 0x7fce7808cd10 con 0x7fce80146800 2024-04-26T20:27:37.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.592+0000 7fce7ea00640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 0x7fce6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:37.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.593+0000 7fce7ea00640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 0x7fce6006edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:37.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.597+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce78092020 con 0x7fce80146800 2024-04-26T20:27:37.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:37.802+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce800d07d0 con 0x7fce80146800 2024-04-26T20:27:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.662+0000 7fce76a00640 1 -- 172.21.15.43:0/3811271893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fce78014030 con 0x7fce80146800 2024-04-26T20:27:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:27:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 msgr2=0x7fce6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 0x7fce6006edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).stop 2024-04-26T20:27:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 msgr2=0x7fce8013e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fce7800cca0 tx=0x7fce7800a650 comp rx=0 tx=0).stop 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 shutdown_connections 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fce6006c8f0 0x7fce6006edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce80146800 0x7fce8013e5e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce8006fec0 0x7fce8013e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 --2- 172.21.15.43:0/3811271893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce8006f230 0x7fce8015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.667+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 >> 172.21.15.43:0/3811271893 conn(0x7fce800b2790 msgr2=0x7fce800b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.668+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 shutdown_connections 2024-04-26T20:27:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.669+0000 7fce85600640 1 -- 172.21.15.43:0/3811271893 wait complete. 2024-04-26T20:27:38.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:27:38.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.912+0000 7fdd89a00640 1 Processor -- start 2024-04-26T20:27:38.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.912+0000 7fdd89a00640 1 -- start start 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd84146800 0x7fdd840cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd8406fec0 0x7fdd8406ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406f230 0x7fdd84071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdd84146be0 con 0x7fdd8406fec0 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdd8406e860 con 0x7fdd84146800 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.914+0000 7fdd89a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8406df30 con 0x7fdd8406f230 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd8406fec0 0x7fdd8406ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd8406fec0 0x7fdd8406ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52088/0 (socket says 172.21.15.43:52088) 2024-04-26T20:27:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 -- 172.21.15.43:0/795081632 learned_addr learned my addr 172.21.15.43:0/795081632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd83400640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406f230 0x7fdd84071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82a00640 1 --2- 172.21.15.43:0/795081632 >> v2:172.21.15.164:3300/0 conn(0x7fdd84146800 0x7fdd840cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 -- 172.21.15.43:0/795081632 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406f230 msgr2=0x7fdd84071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:38.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406f230 0x7fdd84071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 -- 172.21.15.43:0/795081632 >> v2:172.21.15.164:3300/0 conn(0x7fdd84146800 msgr2=0x7fdd840cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:38.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 --2- 172.21.15.43:0/795081632 >> v2:172.21.15.164:3300/0 conn(0x7fdd84146800 0x7fdd840cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.915+0000 7fdd82000640 1 -- 172.21.15.43:0/795081632 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8406e210 con 0x7fdd8406fec0 2024-04-26T20:27:38.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.917+0000 7fdd82000640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406fec0 0x7fdd8406ecf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdd84074610 tx=0x7fdd7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdbfdee008a8bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:38.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.917+0000 7fdd81600640 1 -- 172.21.15.43:0/795081632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037070 con 0x7fdd8406fec0 2024-04-26T20:27:38.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.917+0000 7fdd81600640 1 -- 172.21.15.43:0/795081632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd70032440 con 0x7fdd8406fec0 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.917+0000 7fdd81600640 1 -- 172.21.15.43:0/795081632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032810 con 0x7fdd8406fec0 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 -- 172.21.15.43:0/795081632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406fec0 msgr2=0x7fdd8406ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406fec0 0x7fdd8406ecf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdd84074610 tx=0x7fdd7002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 -- 172.21.15.43:0/795081632 shutdown_connections 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406f230 0x7fdd84071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 --2- 172.21.15.43:0/795081632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406fec0 0x7fdd8406ecf0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 --2- 172.21.15.43:0/795081632 >> v2:172.21.15.164:3300/0 conn(0x7fdd84146800 0x7fdd840cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 -- 172.21.15.43:0/795081632 >> 172.21.15.43:0/795081632 conn(0x7fdd840b2790 msgr2=0x7fdd840b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 -- 172.21.15.43:0/795081632 shutdown_connections 2024-04-26T20:27:38.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.918+0000 7fdd89a00640 1 -- 172.21.15.43:0/795081632 wait complete. 2024-04-26T20:27:38.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.920+0000 7fdd89a00640 1 Processor -- start 2024-04-26T20:27:38.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.920+0000 7fdd89a00640 1 -- start start 2024-04-26T20:27:38.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406fec0 0x7fdd841e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 0x7fdd840aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd840d0520 con 0x7fdd8406f230 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd840d0210 con 0x7fdd84146800 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd89a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdd840723a0 con 0x7fdd8406fec0 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd82a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd82a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52092/0 (socket says 172.21.15.43:52092) 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.921+0000 7fdd82a00640 1 -- 172.21.15.43:0/970441153 learned_addr learned my addr 172.21.15.43:0/970441153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd83400640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 0x7fdd840aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82000640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406fec0 0x7fdd841e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 -- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406fec0 msgr2=0x7fdd841e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406fec0 0x7fdd841e68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 -- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 msgr2=0x7fdd840aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 0x7fdd840aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 -- 172.21.15.43:0/970441153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd841e6fc0 con 0x7fdd8406f230 2024-04-26T20:27:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd83400640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 0x7fdd840aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.922+0000 7fdd82a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400d5f0 tx=0x7fdd7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.923+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740125f0 con 0x7fdd8406f230 2024-04-26T20:27:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.923+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd74019070 con 0x7fdd8406f230 2024-04-26T20:27:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.923+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd740129d0 con 0x7fdd8406f230 2024-04-26T20:27:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.923+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd841e72a0 con 0x7fdd8406f230 2024-04-26T20:27:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.923+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd841e7860 con 0x7fdd8406f230 2024-04-26T20:27:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.924+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd84069350 con 0x7fdd8406f230 2024-04-26T20:27:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.925+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd74012b70 con 0x7fdd8406f230 2024-04-26T20:27:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.926+0000 7fdd7aa00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 0x7fdd6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.926+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6420+0+0 (secure 0 0 0) 0x7fdd7408c280 con 0x7fdd8406f230 2024-04-26T20:27:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.926+0000 7fdd82000640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 0x7fdd6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.927+0000 7fdd82000640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 0x7fdd6406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd70034000 tx=0x7fdd7002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:38.930+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd7405a340 con 0x7fdd8406f230 2024-04-26T20:27:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.141+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdd84073b10 con 0x7fdd8406f230 2024-04-26T20:27:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.684+0000 7fdd7aa00640 1 -- 172.21.15.43:0/970441153 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7fdd74059c40 con 0x7fdd8406f230 2024-04-26T20:27:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 msgr2=0x7fdd6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 0x7fdd6406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdd70034000 tx=0x7fdd7002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 msgr2=0x7fdd841f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400d5f0 tx=0x7fdd7400dab0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 shutdown_connections 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd6406c8f0 0x7fdd6406edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd8406fec0 0x7fdd841e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd84146800 0x7fdd840aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 --2- 172.21.15.43:0/970441153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd8406f230 0x7fdd841f9150 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 >> 172.21.15.43:0/970441153 conn(0x7fdd840b2790 msgr2=0x7fdd8415d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 shutdown_connections 2024-04-26T20:27:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.688+0000 7fdd89a00640 1 -- 172.21.15.43:0/970441153 wait complete. 2024-04-26T20:27:39.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T20:27:39.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.917+0000 7f16bfe00640 1 Processor -- start 2024-04-26T20:27:39.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.917+0000 7f16bfe00640 1 -- start start 2024-04-26T20:27:39.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c006db20 0x7f16c00d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bfe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16c00d13d0 0x7f16c00cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bfe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c01469d0 con 0x7f16c01465f0 2024-04-26T20:27:39.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bfe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f16c006e090 con 0x7f16c00d13d0 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bfe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f16c006e370 con 0x7f16c006db20 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bea00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16c01465f0 0x7f16c015ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58846/0 (socket says 172.21.15.43:58846) 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bea00640 1 -- 172.21.15.43:0/2187832725 learned_addr learned my addr 172.21.15.43:0/2187832725 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16bf400640 1 --2- 172.21.15.43:0/2187832725 >> v2:172.21.15.43:3300/0 conn(0x7f16c00d13d0 0x7f16c00cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.920+0000 7f16b6000640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c006db20 0x7f16c00d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16be000640 1 -- 172.21.15.43:0/2187832725 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903182797 0 0) 0x7f16c01469d0 con 0x7f16c01465f0 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16be000640 1 -- 172.21.15.43:0/2187832725 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16a4003520 con 0x7f16c01465f0 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16bf400640 1 -- 172.21.15.43:0/2187832725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c006db20 msgr2=0x7f16c00d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16bf400640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c006db20 0x7f16c00d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16bf400640 1 -- 172.21.15.43:0/2187832725 >> v1:172.21.15.43:6789/0 conn(0x7f16c01465f0 legacy=0x7f16c015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.921+0000 7f16bf400640 1 -- 172.21.15.43:0/2187832725 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c006e650 con 0x7f16c00d13d0 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.922+0000 7f16bf400640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c00cf560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009d90 tx=0x7f16b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6e91e0029d2f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.922+0000 7f16be000640 1 -- 172.21.15.43:0/2187832725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8007740 con 0x7f16c00d13d0 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.922+0000 7f16be000640 1 -- 172.21.15.43:0/2187832725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b80078e0 con 0x7f16c00d13d0 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.922+0000 7f16be000640 1 -- 172.21.15.43:0/2187832725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8007cb0 con 0x7f16c00d13d0 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 -- 172.21.15.43:0/2187832725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 msgr2=0x7f16c00cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c00cf560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009d90 tx=0x7f16b802a660 comp rx=0 tx=0).stop 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 -- 172.21.15.43:0/2187832725 shutdown_connections 2024-04-26T20:27:39.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c00cf560 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 --2- 172.21.15.43:0/2187832725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c006db20 0x7f16c00d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 -- 172.21.15.43:0/2187832725 >> 172.21.15.43:0/2187832725 conn(0x7f16c00684b0 msgr2=0x7f16c00688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 -- 172.21.15.43:0/2187832725 shutdown_connections 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.923+0000 7f16bfe00640 1 -- 172.21.15.43:0/2187832725 wait complete. 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.925+0000 7f16bfe00640 1 Processor -- start 2024-04-26T20:27:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.925+0000 7f16bfe00640 1 -- start start 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16c006db20 0x7f16c01f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 0x7f16c0071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f16c00b3b20 con 0x7f16c00d13d0 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f16c00cefd0 con 0x7f16c006db20 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.926+0000 7f16bfe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f16c00b3ca0 con 0x7f16c01465f0 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52116/0 (socket says 172.21.15.43:52116) 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 -- 172.21.15.43:0/3802772610 learned_addr learned my addr 172.21.15.43:0/3802772610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16bf400640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 0x7f16c0071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16bea00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16c006db20 0x7f16c01f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 -- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 msgr2=0x7f16c0071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 0x7f16c0071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 -- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16c006db20 msgr2=0x7f16c01f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16c006db20 0x7f16c01f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16b6000640 1 -- 172.21.15.43:0/3802772610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c01f52a0 con 0x7f16c00d13d0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.927+0000 7f16bf400640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 0x7f16c0071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16b6000640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d800 tx=0x7f16a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800b6c0 con 0x7f16c00d13d0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a8019070 con 0x7f16c00d13d0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c01f5580 con 0x7f16c00d13d0 2024-04-26T20:27:39.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800ba80 con 0x7f16c00d13d0 2024-04-26T20:27:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.928+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c0158b70 con 0x7f16c00d13d0 2024-04-26T20:27:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.930+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c0069350 con 0x7f16c00d13d0 2024-04-26T20:27:39.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.930+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16a800bc20 con 0x7f16c00d13d0 2024-04-26T20:27:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.931+0000 7f16b7400640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.931+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f16a808c050 con 0x7f16c00d13d0 2024-04-26T20:27:39.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.932+0000 7f16bea00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:39.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.932+0000 7f16bea00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 0x7f169c06edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16c00af900 tx=0x7f16b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:39.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:39.936+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16a805a480 con 0x7f16c00d13d0 2024-04-26T20:27:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.138+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16c006eec0 con 0x7f16c00d13d0 2024-04-26T20:27:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.688+0000 7f16b7400640 1 -- 172.21.15.43:0/3802772610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f16a8059d80 con 0x7f16c00d13d0 2024-04-26T20:27:40.690 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' removed 2024-04-26T20:27:40.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.693+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 msgr2=0x7f169c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.693+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 0x7f169c06edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f16c00af900 tx=0x7f16b0009040 comp rx=0 tx=0).stop 2024-04-26T20:27:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.693+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 msgr2=0x7f16c01f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:40.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.693+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d800 tx=0x7f16a800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 shutdown_connections 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f16c01465f0 0x7f16c0071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16c00d13d0 0x7f16c01f5d60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 --2- 172.21.15.43:0/3802772610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f16c006db20 0x7f16c01f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.694+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 >> 172.21.15.43:0/3802772610 conn(0x7f16c00684b0 msgr2=0x7f16c015fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:40.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.695+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 shutdown_connections 2024-04-26T20:27:40.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.695+0000 7f16bfe00640 1 -- 172.21.15.43:0/3802772610 wait complete. 2024-04-26T20:27:40.722 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:27:40.723 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:27:40.723 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:27:40.723 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:40.723 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:27:40.723 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:27:40.723 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:27:40.724 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:40.724 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:27:40.724 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:27:40.724 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:27:40.724 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:40.724 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:27:40.724 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:27:40.724 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:27:40.724 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:27:40.744 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:27:40.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:27:40.746 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:27:40.746 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:27:40.747 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:27:40.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:27:40.750 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:27:40.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:27:40.792 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:27:40.792 DEBUG:teuthology.parallel:result is None 2024-04-26T20:27:40.797 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:27:40.797 DEBUG:teuthology.parallel:result is None 2024-04-26T20:27:40.797 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:27:40.798 DEBUG:teuthology.parallel:result is None 2024-04-26T20:27:40.804 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:27:40.804 DEBUG:teuthology.parallel:result is None 2024-04-26T20:27:40.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:27:40.950 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-26T20:27:40.951 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-26T20:27:40.954 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-04-26T20:27:40.955 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-04-26T20:27:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.990+0000 7fc1ce400640 1 Processor -- start 2024-04-26T20:27:40.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.990+0000 7fc1ce400640 1 -- start start 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1c814ceb0 0x7fc1c80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc1c80cfc90 0x7fc1c806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c814d290 con 0x7fc1c814ceb0 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c80b5140 con 0x7fc1c80cfc90 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.991+0000 7fc1ce400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c80b5420 con 0x7fc1c80d1570 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1c814ceb0 0x7fc1c80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:40.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1c814ceb0 0x7fc1c80b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52158/0 (socket says 172.21.15.43:52158) 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 -- 172.21.15.43:0/1522239725 learned_addr learned my addr 172.21.15.43:0/1522239725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c6a00640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7e00640 1 --2- 172.21.15.43:0/1522239725 >> v2:172.21.15.164:3300/0 conn(0x7fc1c80cfc90 0x7fc1c806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 -- 172.21.15.43:0/1522239725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 msgr2=0x7fc1c80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 -- 172.21.15.43:0/1522239725 >> v2:172.21.15.164:3300/0 conn(0x7fc1c80cfc90 msgr2=0x7fc1c806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 --2- 172.21.15.43:0/1522239725 >> v2:172.21.15.164:3300/0 conn(0x7fc1c80cfc90 0x7fc1c806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.992+0000 7fc1c7400640 1 -- 172.21.15.43:0/1522239725 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c80b5700 con 0x7fc1c814ceb0 2024-04-26T20:27:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.993+0000 7fc1c7400640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80b4a60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8166f50 tx=0x7fc1b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24bad7d23a9d0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1c6000640 1 -- 172.21.15.43:0/1522239725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037070 con 0x7fc1c814ceb0 2024-04-26T20:27:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1c6000640 1 -- 172.21.15.43:0/1522239725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8032440 con 0x7fc1c814ceb0 2024-04-26T20:27:40.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1c6000640 1 -- 172.21.15.43:0/1522239725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8032810 con 0x7fc1c814ceb0 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 -- 172.21.15.43:0/1522239725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 msgr2=0x7fc1c80b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80b4a60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8166f50 tx=0x7fc1b802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 -- 172.21.15.43:0/1522239725 shutdown_connections 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 --2- 172.21.15.43:0/1522239725 >> v2:172.21.15.164:3300/0 conn(0x7fc1c80cfc90 0x7fc1c806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 --2- 172.21.15.43:0/1522239725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80b4a60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 -- 172.21.15.43:0/1522239725 >> 172.21.15.43:0/1522239725 conn(0x7fc1c80b2900 msgr2=0x7fc1c80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:40.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 -- 172.21.15.43:0/1522239725 shutdown_connections 2024-04-26T20:27:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.994+0000 7fc1ce400640 1 -- 172.21.15.43:0/1522239725 wait complete. 2024-04-26T20:27:40.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.996+0000 7fc1ce400640 1 Processor -- start 2024-04-26T20:27:40.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.996+0000 7fc1ce400640 1 -- start start 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c80cfc90 0x7fc1c80af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80d19d0 con 0x7fc1c814ceb0 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80b5cc0 con 0x7fc1c80cfc90 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.997+0000 7fc1ce400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c806e530 con 0x7fc1c80d1570 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52162/0 (socket says 172.21.15.43:52162) 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c7e00640 1 -- 172.21.15.43:0/763415701 learned_addr learned my addr 172.21.15.43:0/763415701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c7400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c80cfc90 0x7fc1c80af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 -- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c80cfc90 msgr2=0x7fc1c80af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c80cfc90 0x7fc1c80af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 -- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 msgr2=0x7fc1c80c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c6a00640 1 -- 172.21.15.43:0/763415701 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c80ac990 con 0x7fc1c80d1570 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.998+0000 7fc1c7e00640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1c6a00640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cc770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400e850 tx=0x7fc1b400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4017070 con 0x7fc1c80d1570 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b400c940 con 0x7fc1c80d1570 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400cd10 con 0x7fc1c80d1570 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c80acc70 con 0x7fc1c80d1570 2024-04-26T20:27:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:40.999+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c80ad230 con 0x7fc1c80d1570 2024-04-26T20:27:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.000+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c8069350 con 0x7fc1c80d1570 2024-04-26T20:27:41.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.001+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1b401a070 con 0x7fc1c80d1570 2024-04-26T20:27:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.002+0000 7fc1bf400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.002+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7fc1b408bc40 con 0x7fc1c80d1570 2024-04-26T20:27:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.003+0000 7fc1c7400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.003+0000 7fc1c7400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 0x7fc1a806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.010+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1b4059af0 con 0x7fc1c80d1570 2024-04-26T20:27:41.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.228+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc1c80b4d00 con 0x7fc1c80d1570 2024-04-26T20:27:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.729+0000 7fc1bf400640 1 -- 172.21.15.43:0/763415701 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7fc1b405ddf0 con 0x7fc1c80d1570 2024-04-26T20:27:41.731 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-26T20:27:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 msgr2=0x7fc1a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 0x7fc1a806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 msgr2=0x7fc1c80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cc770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400e850 tx=0x7fc1b400ed10 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 shutdown_connections 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc1a806c8f0 0x7fc1a806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc1c80d1570 0x7fc1c80cc770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c814ceb0 0x7fc1c80c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 --2- 172.21.15.43:0/763415701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1c80cfc90 0x7fc1c80af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 >> 172.21.15.43:0/763415701 conn(0x7fc1c80b2900 msgr2=0x7fc1c8074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:41.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.733+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 shutdown_connections 2024-04-26T20:27:41.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.734+0000 7fc1ce400640 1 -- 172.21.15.43:0/763415701 wait complete. 2024-04-26T20:27:41.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:27:41.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.957+0000 7fcf1ae00640 1 Processor -- start 2024-04-26T20:27:41.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.958+0000 7fcf1ae00640 1 -- start start 2024-04-26T20:27:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.959+0000 7fcf1ae00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.959+0000 7fcf1ae00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf14162880 0x7fcf140cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1ae00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf140cf860 con 0x7fcf140d1d90 2024-04-26T20:27:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1ae00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcf140d3090 con 0x7fcf14162880 2024-04-26T20:27:41.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1ae00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcf140d5b50 con 0x7fcf140d3480 2024-04-26T20:27:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1a400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf14162880 0x7fcf140cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1a400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf14162880 0x7fcf140cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52178/0 (socket says 172.21.15.43:52178) 2024-04-26T20:27:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf1a400640 1 -- 172.21.15.43:0/3084916978 learned_addr learned my addr 172.21.15.43:0/3084916978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.960+0000 7fcf19a00640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.961+0000 7fcf1a400640 1 -- 172.21.15.43:0/3084916978 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 msgr2=0x7fcf140c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.961+0000 7fcf1a400640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.961+0000 7fcf1a400640 1 -- 172.21.15.43:0/3084916978 >> v1:172.21.15.164:6789/0 conn(0x7fcf140d1d90 legacy=0x7fcf14162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.961+0000 7fcf1a400640 1 -- 172.21.15.43:0/3084916978 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf140cfb00 con 0x7fcf14162880 2024-04-26T20:27:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.961+0000 7fcf19a00640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:41.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.962+0000 7fcf1a400640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf14162880 0x7fcf140cffe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009870 tx=0x7fcf0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d3c3ea0e7b2adb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.962+0000 7fcf13e00640 1 -- 172.21.15.43:0/3084916978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c037070 con 0x7fcf14162880 2024-04-26T20:27:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.962+0000 7fcf13e00640 1 -- 172.21.15.43:0/3084916978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c032440 con 0x7fcf14162880 2024-04-26T20:27:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.962+0000 7fcf13e00640 1 -- 172.21.15.43:0/3084916978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c032810 con 0x7fcf14162880 2024-04-26T20:27:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3084916978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf14162880 msgr2=0x7fcf140cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf14162880 0x7fcf140cffe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009870 tx=0x7fcf0c02a270 comp rx=0 tx=0).stop 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3084916978 shutdown_connections 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf14162880 0x7fcf140cffe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3084916978 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3084916978 >> 172.21.15.43:0/3084916978 conn(0x7fcf140d4c50 msgr2=0x7fcf140d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3084916978 shutdown_connections 2024-04-26T20:27:41.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.963+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3084916978 wait complete. 2024-04-26T20:27:41.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.965+0000 7fcf1ae00640 1 Processor -- start 2024-04-26T20:27:41.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.965+0000 7fcf1ae00640 1 -- start start 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf14162880 0x7fcf140b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140d5770 con 0x7fcf140d1d90 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140d2db0 con 0x7fcf14162880 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf1ae00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcf140d58f0 con 0x7fcf140d3480 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52186/0 (socket says 172.21.15.43:52186) 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 -- 172.21.15.43:0/3281663649 learned_addr learned my addr 172.21.15.43:0/3281663649 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:41.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 -- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 msgr2=0x7fcf140b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 -- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf14162880 msgr2=0x7fcf140b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf14162880 0x7fcf140b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.966+0000 7fcf19a00640 1 -- 172.21.15.43:0/3281663649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf140b8940 con 0x7fcf140d1d90 2024-04-26T20:27:41.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf19a00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007fa0 tx=0x7fcf0800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0801a070 con 0x7fcf140d1d90 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf08003b90 con 0x7fcf140d1d90 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004030 con 0x7fcf140d1d90 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf140b8c20 con 0x7fcf140d1d90 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.967+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf140cc250 con 0x7fcf140d1d90 2024-04-26T20:27:41.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.968+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf14069350 con 0x7fcf140d1d90 2024-04-26T20:27:41.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.969+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf0801d070 con 0x7fcf140d1d90 2024-04-26T20:27:41.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.970+0000 7fcf11600640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 0x7fcef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:41.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.971+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fcf0808bbb0 con 0x7fcf140d1d90 2024-04-26T20:27:41.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.971+0000 7fcf19000640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 0x7fcef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:41.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.971+0000 7fcf19000640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 0x7fcef406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f60 tx=0x7fcf000079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:41.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:41.975+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf08010070 con 0x7fcf140d1d90 2024-04-26T20:27:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.261+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcf140cb580 con 0x7fcf140d1d90 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.262+0000 7fcf11600640 1 -- 172.21.15.43:0/3281663649 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fcf08059b40 con 0x7fcf140d1d90 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbDSxmhWZpOhAAWb0QM3LzElq9SPNfKOrsog==", 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.264 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCcDSxmtuzrGBAA0cLhcnn8XVAs9yI4IKlltQ==", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCcDSxmiF6yChAAOPapCewm05UD0aJ8Kzfhuw==", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCcDSxmq9ogJxAAueNEX+99za8dVVuMEfLHyA==", 2024-04-26T20:27:42.266 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCeDSxmteYwOxAA22ydc3JzdGzvj0odfrV7cw==", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCiDSxmQ8d2DhAAo+TieWbjX/TLyv9qRUhcaQ==", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQClDSxmhs7SGhAAuOsUyG/6tvFoGJz4XlTqEw==", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.268 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCoDSxmRj4fJxAATdBuaIMmgSayf1Tspziw4w==", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCuDSxmrGMJOhAAMxMBzTTKWZ1U9MZUuTgSWQ==", 2024-04-26T20:27:42.269 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCyDSxmsRiTCBAA4QL+1v1n797ljg+aHww4uw==", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC1DSxmC+PbIRAAaa7nrIFlTNyeXb9PJ/mu6Q==", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQC4DSxmbBQ5LRAAtipBsb3oO5d/taQd74X/6w==", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-26T20:27:42.271 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCcDSxmFcmKMxAAcYLYOImoPh7zyitYMx2TqQ==", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCdDSxmku1jChAAgPKwhMGf35y3TYCNhG76bQ==", 2024-04-26T20:27:42.272 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCdDSxm8aefABAAhNWzCxvQ5P2w4Qa+4AFTPA==", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-26T20:27:42.273 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCdDSxmIQNZExAAlIxAskjGH8Hq83Xxl0cBJw==", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCaDSxm4fYrOhAAqANw7ERrPLoJQ7jiXby8OQ==", 2024-04-26T20:27:42.274 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxmjqp8JBAARKnY4zqE+DrikfbEt17GyA==", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T20:27:42.275 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxmKMZ8JBAABWcMsscGXSYH1KNe0czhUA==", 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxmiuF8JBAAul402J8WAMKQb11Vn1E82A==", 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxmlft8JBAAkKXs7xPkSE1RchNw8sqPfg==", 2024-04-26T20:27:42.276 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxm/BR9JBAAJj8GV36lEBpW7LAeizzQvQ==", 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDBDSxmcS59JBAAvAX0vvyyZZTG13ukM1LkFg==", 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbDSxmuBUJMhAAdVniffQSTS7cMSX18fHwuA==", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQCbDSxmihfQJhAAxgMUNrLZn63yFfwNrK+4wQ==", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-26T20:27:42.278 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-26T20:27:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-26T20:27:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:42.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-26T20:27:42.279 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 msgr2=0x7fcef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 0x7fcef406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f60 tx=0x7fcf000079b0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 msgr2=0x7fcf1414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007fa0 tx=0x7fcf0800d560 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 shutdown_connections 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcef406c8f0 0x7fcef406edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf140d3480 0x7fcf140b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf14162880 0x7fcf140b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 --2- 172.21.15.43:0/3281663649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf140d1d90 0x7fcf1414d0e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.265+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 >> 172.21.15.43:0/3281663649 conn(0x7fcf140d4c50 msgr2=0x7fcf141607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.266+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 shutdown_connections 2024-04-26T20:27:42.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.266+0000 7fcf1ae00640 1 -- 172.21.15.43:0/3281663649 wait complete. 2024-04-26T20:27:42.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T20:27:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.503+0000 7fdfda000640 1 Processor -- start 2024-04-26T20:27:42.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.503+0000 7fdfda000640 1 -- start start 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfda000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfd4072fb0 0x7fdfd4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfda000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdfd40b4000 0x7fdfd40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfda000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd415fa00 con 0x7fdfd41564b0 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfda000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd4156890 con 0x7fdfd4072fb0 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfda000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd415fca0 con 0x7fdfd40b4000 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfd3400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdfd41564b0 0x7fdfd415ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58864/0 (socket says 172.21.15.43:58864) 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfd3400640 1 -- 172.21.15.43:0/501515553 learned_addr learned my addr 172.21.15.43:0/501515553 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:42.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.505+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/501515553 >> v2:172.21.15.43:3300/0 conn(0x7fdfd4072fb0 0x7fdfd4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.506+0000 7fdfd3e00640 1 --2- 172.21.15.43:0/501515553 >> v2:172.21.15.164:3300/0 conn(0x7fdfd40b4000 0x7fdfd40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.506+0000 7fdfd2a00640 1 -- 172.21.15.43:0/501515553 >> v2:172.21.15.164:3300/0 conn(0x7fdfd40b4000 msgr2=0x7fdfd40b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.506+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/501515553 >> v2:172.21.15.164:3300/0 conn(0x7fdfd40b4000 0x7fdfd40b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.506+0000 7fdfd2a00640 1 -- 172.21.15.43:0/501515553 >> v1:172.21.15.43:6789/0 conn(0x7fdfd41564b0 legacy=0x7fdfd415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.506+0000 7fdfd2a00640 1 -- 172.21.15.43:0/501515553 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd415ff80 con 0x7fdfd4072fb0 2024-04-26T20:27:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.507+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/501515553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd4072fb0 0x7fdfd4073440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406faf0 tx=0x7fdfc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03d1524337d50f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.507+0000 7fdfd2000640 1 -- 172.21.15.43:0/501515553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0037070 con 0x7fdfd4072fb0 2024-04-26T20:27:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.507+0000 7fdfd2000640 1 -- 172.21.15.43:0/501515553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc0032440 con 0x7fdfd4072fb0 2024-04-26T20:27:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.507+0000 7fdfd2000640 1 -- 172.21.15.43:0/501515553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0032810 con 0x7fdfd4072fb0 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 -- 172.21.15.43:0/501515553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd4072fb0 msgr2=0x7fdfd4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 --2- 172.21.15.43:0/501515553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd4072fb0 0x7fdfd4073440 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406faf0 tx=0x7fdfc002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 -- 172.21.15.43:0/501515553 shutdown_connections 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 --2- 172.21.15.43:0/501515553 >> v2:172.21.15.164:3300/0 conn(0x7fdfd40b4000 0x7fdfd40b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 --2- 172.21.15.43:0/501515553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd4072fb0 0x7fdfd4073440 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 -- 172.21.15.43:0/501515553 >> 172.21.15.43:0/501515553 conn(0x7fdfd40682a0 msgr2=0x7fdfd40686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 -- 172.21.15.43:0/501515553 shutdown_connections 2024-04-26T20:27:42.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.508+0000 7fdfda000640 1 -- 172.21.15.43:0/501515553 wait complete. 2024-04-26T20:27:42.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.510+0000 7fdfda000640 1 Processor -- start 2024-04-26T20:27:42.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.510+0000 7fdfda000640 1 -- start start 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 0x7fdfd41417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 0x7fdfd415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4160820 con 0x7fdfd40b4000 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4160540 con 0x7fdfd41564b0 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfda000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd406f540 con 0x7fdfd4072fb0 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52194/0 (socket says 172.21.15.43:52194) 2024-04-26T20:27:42.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.511+0000 7fdfd2a00640 1 -- 172.21.15.43:0/975670633 learned_addr learned my addr 172.21.15.43:0/975670633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd3e00640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 0x7fdfd415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd3400640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 0x7fdfd41417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 -- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 msgr2=0x7fdfd41417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 0x7fdfd41417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 -- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 msgr2=0x7fdfd415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 0x7fdfd415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 -- 172.21.15.43:0/975670633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd4141cf0 con 0x7fdfd40b4000 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd3e00640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 0x7fdfd415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd3400640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 0x7fdfd41417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.512+0000 7fdfd2a00640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.513+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0037040 con 0x7fdfd40b4000 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.513+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc0042070 con 0x7fdfd40b4000 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.513+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0030bd0 con 0x7fdfd40b4000 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.513+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41e7190 con 0x7fdfd40b4000 2024-04-26T20:27:42.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.513+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41e7750 con 0x7fdfd40b4000 2024-04-26T20:27:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.514+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd40b5d60 con 0x7fdfd40b4000 2024-04-26T20:27:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.515+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfc0047460 con 0x7fdfd40b4000 2024-04-26T20:27:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.516+0000 7fdfcb400640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.517+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fdfc00ac5d0 con 0x7fdfd40b4000 2024-04-26T20:27:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.517+0000 7fdfd3400640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:42.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.517+0000 7fdfd3400640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 0x7fdfb406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40012e0 tx=0x7fdfc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.521+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdfd40b5d60 con 0x7fdfd40b4000 2024-04-26T20:27:42.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.809+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fdfd4074160 con 0x7fdfd40b4000 2024-04-26T20:27:42.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.814+0000 7fdfcb400640 1 -- 172.21.15.43:0/975670633 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fdfc0032e20 con 0x7fdfd40b4000 2024-04-26T20:27:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.817+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 msgr2=0x7fdfb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.817+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 0x7fdfb406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40012e0 tx=0x7fdfc4009040 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.817+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 msgr2=0x7fdfd41e61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.817+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc00046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 shutdown_connections 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfd41564b0 0x7fdfd415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd40b4000 0x7fdfd41e61d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 --2- 172.21.15.43:0/975670633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdfd4072fb0 0x7fdfd41417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 >> 172.21.15.43:0/975670633 conn(0x7fdfd40682a0 msgr2=0x7fdfd415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 shutdown_connections 2024-04-26T20:27:42.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:42.818+0000 7fdfda000640 1 -- 172.21.15.43:0/975670633 wait complete. 2024-04-26T20:27:42.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T20:27:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.053+0000 7f91a8a00640 1 Processor -- start 2024-04-26T20:27:43.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.053+0000 7f91a8a00640 1 -- start start 2024-04-26T20:27:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.054+0000 7f91a8a00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f91a415b900 0x7f91a40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.054+0000 7f91a8a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a4071d20 con 0x7f91a406f840 2024-04-26T20:27:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.054+0000 7f91a8a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a40b4f00 con 0x7f91a415feb0 2024-04-26T20:27:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.054+0000 7f91a8a00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f91a40b51e0 con 0x7f91a415b900 2024-04-26T20:27:43.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.055+0000 7f91a1e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91a406f840 0x7f91a415f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58866/0 (socket says 172.21.15.43:58866) 2024-04-26T20:27:43.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.055+0000 7f91a1e00640 1 -- 172.21.15.43:0/168156813 learned_addr learned my addr 172.21.15.43:0/168156813 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:43.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.055+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406169450 0 0) 0x7f91a4071d20 con 0x7f91a406f840 2024-04-26T20:27:43.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.055+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9188003520 con 0x7f91a406f840 2024-04-26T20:27:43.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.055+0000 7f91a2800640 1 --2- 172.21.15.43:0/168156813 >> v2:172.21.15.164:3300/0 conn(0x7f91a415b900 0x7f91a40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810948222 0 0) 0x7f9188003520 con 0x7f91a406f840 2024-04-26T20:27:43.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 >> v2:172.21.15.164:3300/0 conn(0x7f91a415b900 msgr2=0x7f91a40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 --2- 172.21.15.43:0/168156813 >> v2:172.21.15.164:3300/0 conn(0x7f91a415b900 0x7f91a40b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 >> v1:172.21.15.164:6789/0 conn(0x7f91a415feb0 legacy=0x7f91a415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a4071f90 con 0x7f91a406f840 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f9190001670 con 0x7f91a406f840 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 >> v1:172.21.15.43:6789/0 conn(0x7f91a406f840 legacy=0x7f91a415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91880039b0 0x7f9188003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91a40b51e0 con 0x7f9188004800 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f918800c0f0 con 0x7f91880039b0 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f918800c2f0 con 0x7f91880051c0 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.056+0000 7f91a1e00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a2800640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91880039b0 0x7f9188003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a1e00640 1 -- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91880051c0 msgr2=0x7f918800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a3200640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a1e00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a1e00640 1 -- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91880039b0 msgr2=0x7f9188003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a1e00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91880039b0 0x7f9188003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a1e00640 1 -- 172.21.15.43:0/168156813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9188003520 con 0x7f9188004800 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a3200640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a2800640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91880039b0 0x7f9188003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.057+0000 7f91a3200640 1 -- 172.21.15.43:0/168156813 reap_dead start 2024-04-26T20:27:43.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.058+0000 7f91a1e00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f9188004c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f919000c210 tx=0x7f9188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf3aa5bee9da820 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.058+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190007d30 con 0x7f9188004800 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.058+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190005af0 con 0x7f9188004800 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.058+0000 7f91a1400640 1 -- 172.21.15.43:0/168156813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190007070 con 0x7f9188004800 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 -- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 msgr2=0x7f9188004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f9188004c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f919000c210 tx=0x7f9188005830 comp rx=0 tx=0).stop 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 -- 172.21.15.43:0/168156813 shutdown_connections 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 --2- 172.21.15.43:0/168156813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 -- 172.21.15.43:0/168156813 >> 172.21.15.43:0/168156813 conn(0x7f91a40b26b0 msgr2=0x7f91a40b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 -- 172.21.15.43:0/168156813 shutdown_connections 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.059+0000 7f91a8a00640 1 -- 172.21.15.43:0/168156813 wait complete. 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.061+0000 7f91a8a00640 1 Processor -- start 2024-04-26T20:27:43.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.061+0000 7f91a8a00640 1 -- start start 2024-04-26T20:27:43.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91a4150600 0x7f91a40cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a40cf010 0x7f91a40cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415f920 con 0x7f9188004800 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415f7a0 con 0x7f91a40cf010 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a8a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f91a415faa0 con 0x7f91a4150600 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52210/0 (socket says 172.21.15.43:52210) 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a2800640 1 -- 172.21.15.43:0/107822056 learned_addr learned my addr 172.21.15.43:0/107822056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a2800640 1 -- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91a4150600 msgr2=0x7f91a40cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.062+0000 7f91a2800640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91a4150600 0x7f91a40cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a2800640 1 -- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a40cf010 msgr2=0x7f91a40cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a2800640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a40cf010 0x7f91a40cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a2800640 1 -- 172.21.15.43:0/107822056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a414da10 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a2800640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f919400e810 tx=0x7f919400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194017070 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919400c8d0 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919400f040 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a4150e60 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.063+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a40ca0b0 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.065+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4069350 con 0x7f9188004800 2024-04-26T20:27:43.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.066+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f919401a070 con 0x7f9188004800 2024-04-26T20:27:43.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.067+0000 7f919aa00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 0x7f918406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.067+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f919408b630 con 0x7f9188004800 2024-04-26T20:27:43.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.068+0000 7f91a1e00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 0x7f918406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.068+0000 7f91a1e00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 0x7f918406ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9190033000 tx=0x7f9190009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.072+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9194059bc0 con 0x7f9188004800 2024-04-26T20:27:43.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.363+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f91a4156bf0 con 0x7f9188004800 2024-04-26T20:27:43.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.367+0000 7f919aa00640 1 -- 172.21.15.43:0/107822056 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f91940594c0 con 0x7f9188004800 2024-04-26T20:27:43.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.369+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 msgr2=0x7f918406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.369+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 0x7f918406ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9190033000 tx=0x7f9190009030 comp rx=0 tx=0).stop 2024-04-26T20:27:43.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.369+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 msgr2=0x7f91a41500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.369+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f919400e810 tx=0x7f919400ecd0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 shutdown_connections 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f918406c920 0x7f918406ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f91a4150600 0x7f91a40cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f91a40cf010 0x7f91a40cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 --2- 172.21.15.43:0/107822056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9188004800 0x7f91a41500c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 >> 172.21.15.43:0/107822056 conn(0x7f91a40b26b0 msgr2=0x7f91a4073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 shutdown_connections 2024-04-26T20:27:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.370+0000 7f91a8a00640 1 -- 172.21.15.43:0/107822056 wait complete. 2024-04-26T20:27:43.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T20:27:43.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.591+0000 7f6363200640 1 Processor -- start 2024-04-26T20:27:43.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.591+0000 7f6363200640 1 -- start start 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6363200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f636415b900 0x7f63640b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6363200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6363200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6364071d20 con 0x7f636415fce0 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6363200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6364160130 con 0x7f636415b900 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6363200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f63640b4f00 con 0x7f636406f840 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6361e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f636415b900 0x7f63640b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6361e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f636415b900 0x7f63640b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52220/0 (socket says 172.21.15.43:52220) 2024-04-26T20:27:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.593+0000 7f6361e00640 1 -- 172.21.15.43:0/3846295459 learned_addr learned my addr 172.21.15.43:0/3846295459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.594+0000 7f6361e00640 1 -- 172.21.15.43:0/3846295459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 msgr2=0x7f636415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.594+0000 7f6361e00640 1 --2- 172.21.15.43:0/3846295459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.594+0000 7f6361e00640 1 -- 172.21.15.43:0/3846295459 >> v1:172.21.15.164:6789/0 conn(0x7f636415fce0 legacy=0x7f63640724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.594+0000 7f6361e00640 1 -- 172.21.15.43:0/3846295459 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63640b51e0 con 0x7f636415b900 2024-04-26T20:27:43.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.595+0000 7f6361e00640 1 --2- 172.21.15.43:0/3846295459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63640b46d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6354009870 tx=0x7f635402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b55cd944f00c81e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.595+0000 7f6361400640 1 -- 172.21.15.43:0/3846295459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354037070 con 0x7f636415b900 2024-04-26T20:27:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.595+0000 7f6361400640 1 -- 172.21.15.43:0/3846295459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354032440 con 0x7f636415b900 2024-04-26T20:27:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.595+0000 7f6361400640 1 -- 172.21.15.43:0/3846295459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354032810 con 0x7f636415b900 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 -- 172.21.15.43:0/3846295459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 msgr2=0x7f63640b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 --2- 172.21.15.43:0/3846295459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63640b46d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6354009870 tx=0x7f635402a210 comp rx=0 tx=0).stop 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 -- 172.21.15.43:0/3846295459 shutdown_connections 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 --2- 172.21.15.43:0/3846295459 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 --2- 172.21.15.43:0/3846295459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63640b46d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 -- 172.21.15.43:0/3846295459 >> 172.21.15.43:0/3846295459 conn(0x7f63640b26b0 msgr2=0x7f63640b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 -- 172.21.15.43:0/3846295459 shutdown_connections 2024-04-26T20:27:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.596+0000 7f6363200640 1 -- 172.21.15.43:0/3846295459 wait complete. 2024-04-26T20:27:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.598+0000 7f6363200640 1 Processor -- start 2024-04-26T20:27:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.598+0000 7f6363200640 1 -- start start 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f636415fce0 0x7f6364147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f63640b54c0 con 0x7f636415b900 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63640b4c10 con 0x7f636415fce0 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.599+0000 7f6363200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f63640b5640 con 0x7f636406f840 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52228/0 (socket says 172.21.15.43:52228) 2024-04-26T20:27:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 -- 172.21.15.43:0/2670030146 learned_addr learned my addr 172.21.15.43:0/2670030146 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6361e00640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 -- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 msgr2=0x7f636415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 -- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f636415fce0 msgr2=0x7f6364147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f636415fce0 0x7f6364147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.600+0000 7f6359400640 1 -- 172.21.15.43:0/2670030146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6364158c40 con 0x7f636415b900 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f6359400640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f634c00b4a0 tx=0x7f634c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c014070 con 0x7f636415b900 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f634c010610 con 0x7f636415b900 2024-04-26T20:27:43.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6364158f20 con 0x7f636415b900 2024-04-26T20:27:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c010a00 con 0x7f636415b900 2024-04-26T20:27:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.601+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63641594e0 con 0x7f636415b900 2024-04-26T20:27:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.602+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6364069350 con 0x7f636415b900 2024-04-26T20:27:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.603+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f634c017070 con 0x7f636415b900 2024-04-26T20:27:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.604+0000 7f635aa00640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.604+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f634c08b420 con 0x7f636415b900 2024-04-26T20:27:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.605+0000 7f6361e00640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:43.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.605+0000 7f6361e00640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 0x7f634006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6354034000 tx=0x7f635402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:43.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.609+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f634c059a10 con 0x7f636415b900 2024-04-26T20:27:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.892+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f63640b4570 con 0x7f636415b900 2024-04-26T20:27:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.896+0000 7f635aa00640 1 -- 172.21.15.43:0/2670030146 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f634c059310 con 0x7f636415b900 2024-04-26T20:27:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 msgr2=0x7f634006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 0x7f634006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6354034000 tx=0x7f635402e000 comp rx=0 tx=0).stop 2024-04-26T20:27:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 msgr2=0x7f63641584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f634c00b4a0 tx=0x7f634c00b960 comp rx=0 tx=0).stop 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 shutdown_connections 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f634006c8f0 0x7f634006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f636415fce0 0x7f6364147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f636415b900 0x7f63641584a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 --2- 172.21.15.43:0/2670030146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f636406f840 0x7f636415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:43.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 >> 172.21.15.43:0/2670030146 conn(0x7f63640b26b0 msgr2=0x7f63640d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:43.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.899+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 shutdown_connections 2024-04-26T20:27:43.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:43.900+0000 7f6363200640 1 -- 172.21.15.43:0/2670030146 wait complete. 2024-04-26T20:27:43.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T20:27:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.134+0000 7f1a67e00640 1 Processor -- start 2024-04-26T20:27:44.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.134+0000 7f1a67e00640 1 -- start start 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.135+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 0x7f1a6815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.135+0000 7f1a67e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1a6806f840 0x7f1a6815fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.135+0000 7f1a67e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a68071d20 con 0x7f1a6815b900 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.135+0000 7f1a67e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1a680b4f00 con 0x7f1a6806f840 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.135+0000 7f1a67e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1a680b51e0 con 0x7f1a680734c0 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a66a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a6815b900 0x7f1a680b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58882/0 (socket says 172.21.15.43:58882) 2024-04-26T20:27:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a66a00640 1 -- 172.21.15.43:0/66398371 learned_addr learned my addr 172.21.15.43:0/66398371 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:44.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950697560 0 0) 0x7f1a68071d20 con 0x7f1a6815b900 2024-04-26T20:27:44.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a4c003520 con 0x7f1a6815b900 2024-04-26T20:27:44.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a66000640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 0x7f1a6815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.136+0000 7f1a67400640 1 --2- 172.21.15.43:0/66398371 >> v2:172.21.15.164:3300/0 conn(0x7f1a6806f840 0x7f1a6815fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1098815206 0 0) 0x7f1a4c003520 con 0x7f1a6815b900 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 msgr2=0x7f1a6815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 0x7f1a6815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 >> v2:172.21.15.164:3300/0 conn(0x7f1a6806f840 msgr2=0x7f1a6815fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 --2- 172.21.15.43:0/66398371 >> v2:172.21.15.164:3300/0 conn(0x7f1a6806f840 0x7f1a6815fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a68071f90 con 0x7f1a6815b900 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f1a580019f0 con 0x7f1a6815b900 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 >> v1:172.21.15.43:6789/0 conn(0x7f1a6815b900 legacy=0x7f1a680b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039b0 0x7f1a4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c004800 0x7f1a4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 0x7f1a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680b4f00 con 0x7f1a4c0039b0 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a680b51e0 con 0x7f1a4c004800 2024-04-26T20:27:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c00baa0 con 0x7f1a680734c0 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a66a00640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039b0 0x7f1a4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a66a00640 1 -- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 msgr2=0x7f1a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a66a00640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a680734c0 0x7f1a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a66a00640 1 -- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c004800 msgr2=0x7f1a4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.137+0000 7f1a66a00640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c004800 0x7f1a4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.138+0000 7f1a66a00640 1 -- 172.21.15.43:0/66398371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c003520 con 0x7f1a4c0039b0 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a66a00640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039b0 0x7f1a4c003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1a580013d0 tx=0x7f1a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21bd5c09a710536 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800a4c0 con 0x7f1a4c0039b0 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5802f070 con 0x7f1a4c0039b0 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a65600640 1 -- 172.21.15.43:0/66398371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5802bc90 con 0x7f1a4c0039b0 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a67e00640 1 -- 172.21.15.43:0/66398371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039b0 msgr2=0x7f1a4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a67e00640 1 --2- 172.21.15.43:0/66398371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039b0 0x7f1a4c003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1a580013d0 tx=0x7f1a4c005830 comp rx=0 tx=0).stop 2024-04-26T20:27:44.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.139+0000 7f1a67400640 1 -- 172.21.15.43:0/66398371 reap_dead start 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.140+0000 7f1a67e00640 1 -- 172.21.15.43:0/66398371 shutdown_connections 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.140+0000 7f1a67e00640 1 -- 172.21.15.43:0/66398371 >> 172.21.15.43:0/66398371 conn(0x7f1a680b26b0 msgr2=0x7f1a680b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.140+0000 7f1a67e00640 1 -- 172.21.15.43:0/66398371 shutdown_connections 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.140+0000 7f1a67e00640 1 -- 172.21.15.43:0/66398371 wait complete. 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.142+0000 7f1a67e00640 1 Processor -- start 2024-04-26T20:27:44.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.142+0000 7f1a67e00640 1 -- start start 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c0039b0 0x7f1a681683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a68153d80 0x7f1a68164f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6815f920 con 0x7f1a68167630 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6815f7a0 con 0x7f1a4c0039b0 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6815faa0 con 0x7f1a68153d80 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52252/0 (socket says 172.21.15.43:52252) 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 -- 172.21.15.43:0/2150639820 learned_addr learned my addr 172.21.15.43:0/2150639820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a66a00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c0039b0 0x7f1a681683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 -- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a68153d80 msgr2=0x7f1a68164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a68153d80 0x7f1a68164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 -- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c0039b0 msgr2=0x7f1a681683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c0039b0 0x7f1a681683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.143+0000 7f1a67400640 1 -- 172.21.15.43:0/2150639820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681688f0 con 0x7f1a68167630 2024-04-26T20:27:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a67400640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806d490 tx=0x7f1a6000d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60004020 con 0x7f1a68167630 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a600041c0 con 0x7f1a68167630 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a681542f0 con 0x7f1a68167630 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600043b0 con 0x7f1a68167630 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.144+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a68166e30 con 0x7f1a68167630 2024-04-26T20:27:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.145+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68069350 con 0x7f1a68167630 2024-04-26T20:27:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.146+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a6001a020 con 0x7f1a68167630 2024-04-26T20:27:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.147+0000 7f1a5ea00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 0x7f1a4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.147+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1a6008b070 con 0x7f1a68167630 2024-04-26T20:27:44.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.148+0000 7f1a66a00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 0x7f1a4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.148+0000 7f1a66a00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 0x7f1a4806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a58033000 tx=0x7f1a58038040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.152+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a60059650 con 0x7f1a68167630 2024-04-26T20:27:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.427+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1a6806cc60 con 0x7f1a68167630 2024-04-26T20:27:44.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.433+0000 7f1a5ea00640 1 -- 172.21.15.43:0/2150639820 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f1a60058f50 con 0x7f1a68167630 2024-04-26T20:27:44.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 msgr2=0x7f1a4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 0x7f1a4806ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1a58033000 tx=0x7f1a58038040 comp rx=0 tx=0).stop 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 msgr2=0x7f1a68167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806d490 tx=0x7f1a6000d7e0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 shutdown_connections 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1a4806c920 0x7f1a4806ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1a68153d80 0x7f1a68164f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68167630 0x7f1a68167ab0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 --2- 172.21.15.43:0/2150639820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a4c0039b0 0x7f1a681683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 >> 172.21.15.43:0/2150639820 conn(0x7f1a680b26b0 msgr2=0x7f1a6815a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.435+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 shutdown_connections 2024-04-26T20:27:44.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.436+0000 7f1a67e00640 1 -- 172.21.15.43:0/2150639820 wait complete. 2024-04-26T20:27:44.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T20:27:44.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.658+0000 7f1b7be00640 1 Processor -- start 2024-04-26T20:27:44.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.659+0000 7f1b7be00640 1 -- start start 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.660+0000 7f1b7be00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b7c161460 0x7f1b7c160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.660+0000 7f1b7be00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7c0b50e0 con 0x7f1b7c1463e0 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.660+0000 7f1b7be00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7c0b4560 con 0x7f1b7c160d90 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.660+0000 7f1b7be00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c1611d0 con 0x7f1b7c161460 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b7aa00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1b7c1463e0 0x7f1b7c0b58b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58884/0 (socket says 172.21.15.43:58884) 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b7aa00640 1 -- 172.21.15.43:0/1576228289 learned_addr learned my addr 172.21.15.43:0/1576228289 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050921642 0 0) 0x7f1b7c0b50e0 con 0x7f1b7c1463e0 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b60003520 con 0x7f1b7c1463e0 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1576228289 >> v2:172.21.15.164:3300/0 conn(0x7f1b7c161460 0x7f1b7c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1303873631 0 0) 0x7f1b60003520 con 0x7f1b7c1463e0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 >> v2:172.21.15.164:3300/0 conn(0x7f1b7c161460 msgr2=0x7f1b7c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 --2- 172.21.15.43:0/1576228289 >> v2:172.21.15.164:3300/0 conn(0x7f1b7c161460 0x7f1b7c160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.661+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 >> v1:172.21.15.164:6789/0 conn(0x7f1b7c160d90 legacy=0x7f1b7c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c0b4840 con 0x7f1b7c1463e0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f1b6c001670 con 0x7f1b7c1463e0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 >> v1:172.21.15.43:6789/0 conn(0x7f1b7c1463e0 legacy=0x7f1b7c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b600039b0 0x7f1b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b600052e0 0x7f1b6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1611d0 con 0x7f1b60004800 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6000c1b0 con 0x7f1b600052e0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6000c3b0 con 0x7f1b600039b0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 -- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b600039b0 msgr2=0x7f1b60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b600039b0 0x7f1b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 -- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b600052e0 msgr2=0x7f1b6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b600052e0 0x7f1b6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7a000640 1 -- 172.21.15.43:0/1576228289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b60003520 con 0x7f1b60004800 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.662+0000 7f1b7b400640 1 -- 172.21.15.43:0/1576228289 reap_dead start 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.663+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c071a60 tx=0x7f1b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cfcf6baf3943a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68030070 con 0x7f1b60004800 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6802adb0 con 0x7f1b60004800 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b79600640 1 -- 172.21.15.43:0/1576228289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800b610 con 0x7f1b60004800 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b7be00640 1 -- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 msgr2=0x7f1b60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c071a60 tx=0x7f1b60005830 comp rx=0 tx=0).stop 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b7be00640 1 -- 172.21.15.43:0/1576228289 shutdown_connections 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1576228289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b60004800 0x7f1b60004c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.664+0000 7f1b7be00640 1 -- 172.21.15.43:0/1576228289 >> 172.21.15.43:0/1576228289 conn(0x7f1b7c0b26b0 msgr2=0x7f1b7c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.665+0000 7f1b7be00640 1 -- 172.21.15.43:0/1576228289 shutdown_connections 2024-04-26T20:27:44.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.665+0000 7f1b7be00640 1 -- 172.21.15.43:0/1576228289 wait complete. 2024-04-26T20:27:44.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.666+0000 7f1b7be00640 1 Processor -- start 2024-04-26T20:27:44.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.666+0000 7f1b7be00640 1 -- start start 2024-04-26T20:27:44.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b60004800 0x7f1b7c14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c1450f0 0x7f1b7c13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1467c0 con 0x7f1b7c144c90 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c0b4e00 con 0x7f1b7c1450f0 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.667+0000 7f1b7be00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c160850 con 0x7f1b60004800 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52262/0 (socket says 172.21.15.43:52262) 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 -- 172.21.15.43:0/1254767050 learned_addr learned my addr 172.21.15.43:0/1254767050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 -- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b60004800 msgr2=0x7f1b7c14f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b60004800 0x7f1b7c14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 -- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c1450f0 msgr2=0x7f1b7c13bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c1450f0 0x7f1b7c13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 -- 172.21.15.43:0/1254767050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c14eea0 con 0x7f1b7c144c90 2024-04-26T20:27:44.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.668+0000 7f1b7a000640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800bb00 tx=0x7f1b68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.669+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68043020 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.669+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6803f680 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.669+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800b610 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.669+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1f8f70 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.669+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c14f160 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.670+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c069350 con 0x7f1b7c144c90 2024-04-26T20:27:44.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.671+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b68046020 con 0x7f1b7c144c90 2024-04-26T20:27:44.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.672+0000 7f1b73400640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:44.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.672+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f1b680ac990 con 0x7f1b7c144c90 2024-04-26T20:27:44.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.672+0000 7f1b7aa00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:44.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.673+0000 7f1b7aa00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c069a00 tx=0x7f1b6c001b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:44.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.676+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b6804d020 con 0x7f1b7c144c90 2024-04-26T20:27:44.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.957+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f1b7c0b2ea0 con 0x7f1b7c144c90 2024-04-26T20:27:44.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.961+0000 7f1b73400640 1 -- 172.21.15.43:0/1254767050 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f1b6807aee0 con 0x7f1b7c144c90 2024-04-26T20:27:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.964+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 msgr2=0x7f1b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.964+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c069a00 tx=0x7f1b6c001b10 comp rx=0 tx=0).stop 2024-04-26T20:27:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.964+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 msgr2=0x7f1b7c1f84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.964+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800bb00 tx=0x7f1b68004730 comp rx=0 tx=0).stop 2024-04-26T20:27:44.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.964+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 shutdown_connections 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1b5c06c8f0 0x7f1b5c06edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b7c1450f0 0x7f1b7c13bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c144c90 0x7f1b7c1f84b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 --2- 172.21.15.43:0/1254767050 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b60004800 0x7f1b7c14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 >> 172.21.15.43:0/1254767050 conn(0x7f1b7c0b26b0 msgr2=0x7f1b7c1412e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 shutdown_connections 2024-04-26T20:27:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:44.965+0000 7f1b7be00640 1 -- 172.21.15.43:0/1254767050 wait complete. 2024-04-26T20:27:44.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T20:27:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.181+0000 7fe38da00640 1 Processor -- start 2024-04-26T20:27:45.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.181+0000 7fe38da00640 1 -- start start 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.183+0000 7fe38da00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 0x7fe3880b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.183+0000 7fe38da00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.183+0000 7fe38da00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3880b3c90 con 0x7fe3881600b0 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.183+0000 7fe38da00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe3880b4b20 con 0x7fe38815fcd0 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.183+0000 7fe38da00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe3880af8d0 con 0x7fe388072c10 2024-04-26T20:27:45.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.184+0000 7fe38d000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.184+0000 7fe38d000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:42002/0 (socket says 172.21.15.43:42002) 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.184+0000 7fe38d000640 1 -- 172.21.15.43:0/4293242990 learned_addr learned my addr 172.21.15.43:0/4293242990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.184+0000 7fe387e00640 1 --2- 172.21.15.43:0/4293242990 >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 0x7fe3880b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.185+0000 7fe38d000640 1 -- 172.21.15.43:0/4293242990 >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 msgr2=0x7fe3880b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.185+0000 7fe38d000640 1 --2- 172.21.15.43:0/4293242990 >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 0x7fe3880b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.185+0000 7fe38d000640 1 -- 172.21.15.43:0/4293242990 >> v1:172.21.15.164:6789/0 conn(0x7fe3881600b0 legacy=0x7fe388072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.185+0000 7fe38d000640 1 -- 172.21.15.43:0/4293242990 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3880b3f10 con 0x7fe388072c10 2024-04-26T20:27:45.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.185+0000 7fe387e00640 1 --2- 172.21.15.43:0/4293242990 >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 0x7fe3880b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:27:45.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.186+0000 7fe38d000640 1 --2- 172.21.15.43:0/4293242990 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe380009870 tx=0x7fe38002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6740a3666c4e6149 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.186+0000 7fe386a00640 1 -- 172.21.15.43:0/4293242990 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380037070 con 0x7fe388072c10 2024-04-26T20:27:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.186+0000 7fe386a00640 1 -- 172.21.15.43:0/4293242990 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe380032440 con 0x7fe388072c10 2024-04-26T20:27:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.186+0000 7fe386a00640 1 -- 172.21.15.43:0/4293242990 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380032810 con 0x7fe388072c10 2024-04-26T20:27:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 -- 172.21.15.43:0/4293242990 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 msgr2=0x7fe3880b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 --2- 172.21.15.43:0/4293242990 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe380009870 tx=0x7fe38002a250 comp rx=0 tx=0).stop 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 -- 172.21.15.43:0/4293242990 shutdown_connections 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 --2- 172.21.15.43:0/4293242990 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b5a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 --2- 172.21.15.43:0/4293242990 >> v2:172.21.15.164:3300/0 conn(0x7fe38815fcd0 0x7fe3880b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 -- 172.21.15.43:0/4293242990 >> 172.21.15.43:0/4293242990 conn(0x7fe3880b26b0 msgr2=0x7fe3880b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 -- 172.21.15.43:0/4293242990 shutdown_connections 2024-04-26T20:27:45.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.187+0000 7fe38da00640 1 -- 172.21.15.43:0/4293242990 wait complete. 2024-04-26T20:27:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.189+0000 7fe38da00640 1 Processor -- start 2024-04-26T20:27:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.189+0000 7fe38da00640 1 -- start start 2024-04-26T20:27:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3881600b0 0x7fe38814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880b47b0 con 0x7fe38815fcd0 2024-04-26T20:27:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3880b44d0 con 0x7fe3881600b0 2024-04-26T20:27:45.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe38da00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe3880b4fb0 con 0x7fe388072c10 2024-04-26T20:27:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.190+0000 7fe387400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52272/0 (socket says 172.21.15.43:52272) 2024-04-26T20:27:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 -- 172.21.15.43:0/3550335836 learned_addr learned my addr 172.21.15.43:0/3550335836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe38d000640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3881600b0 0x7fe38814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387e00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 -- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 msgr2=0x7fe3880b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 -- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3881600b0 msgr2=0x7fe38814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3881600b0 0x7fe38814a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 -- 172.21.15.43:0/3550335836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3881f57a0 con 0x7fe38815fcd0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.191+0000 7fe387400640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c3c0 tx=0x7fe37000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.192+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700052d0 con 0x7fe38815fcd0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.192+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe370005490 con 0x7fe38815fcd0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.192+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370007260 con 0x7fe38815fcd0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.192+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3881f81b0 con 0x7fe38815fcd0 2024-04-26T20:27:45.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.192+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3881f8770 con 0x7fe38815fcd0 2024-04-26T20:27:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.193+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3881f8970 con 0x7fe38815fcd0 2024-04-26T20:27:45.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.194+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe370007400 con 0x7fe38815fcd0 2024-04-26T20:27:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.195+0000 7fe37fe00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 0x7fe36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.195+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe37008aea0 con 0x7fe38815fcd0 2024-04-26T20:27:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.196+0000 7fe387e00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 0x7fe36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.196+0000 7fe387e00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 0x7fe36406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe378007440 tx=0x7fe37800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.200+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3881f8970 con 0x7fe38815fcd0 2024-04-26T20:27:45.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.486+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fe388069350 con 0x7fe38815fcd0 2024-04-26T20:27:45.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.491+0000 7fe37fe00640 1 -- 172.21.15.43:0/3550335836 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fe3880b3c90 con 0x7fe38815fcd0 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 msgr2=0x7fe36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 0x7fe36406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe378007440 tx=0x7fe37800b040 comp rx=0 tx=0).stop 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 msgr2=0x7fe388149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c3c0 tx=0x7fe37000c880 comp rx=0 tx=0).stop 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 shutdown_connections 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fe36406c8f0 0x7fe36406edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3881600b0 0x7fe38814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe38815fcd0 0x7fe388149f30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 --2- 172.21.15.43:0/3550335836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe388072c10 0x7fe3880b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.493+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 >> 172.21.15.43:0/3550335836 conn(0x7fe3880b26b0 msgr2=0x7fe3880b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:45.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.494+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 shutdown_connections 2024-04-26T20:27:45.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.494+0000 7fe38da00640 1 -- 172.21.15.43:0/3550335836 wait complete. 2024-04-26T20:27:45.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.713+0000 7fa89b200640 1 Processor -- start 2024-04-26T20:27:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.713+0000 7fa89b200640 1 -- start start 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.714+0000 7fa89b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa89c0d1620 0x7fa89c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.714+0000 7fa89b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0d1a00 0x7fa89c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.714+0000 7fa89b200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89c0d0480 con 0x7fa89c0b50b0 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.714+0000 7fa89b200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa89c0d0ce0 con 0x7fa89c0d1620 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.714+0000 7fa89b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c0d0720 con 0x7fa89c0d1a00 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa89c0d1620 0x7fa89c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa89c0d1620 0x7fa89c0d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52288/0 (socket says 172.21.15.43:52288) 2024-04-26T20:27:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 -- 172.21.15.43:0/3809455708 learned_addr learned my addr 172.21.15.43:0/3809455708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 -- 172.21.15.43:0/3809455708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0d1a00 msgr2=0x7fa89c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 --2- 172.21.15.43:0/3809455708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0d1a00 0x7fa89c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 -- 172.21.15.43:0/3809455708 >> v1:172.21.15.43:6789/0 conn(0x7fa89c0b50b0 legacy=0x7fa89c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.715+0000 7fa899e00640 1 -- 172.21.15.43:0/3809455708 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c0d0a00 con 0x7fa89c0d1620 2024-04-26T20:27:45.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.716+0000 7fa899e00640 1 --2- 172.21.15.43:0/3809455708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0d5750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009870 tx=0x7fa88c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ce91df9a5c3f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa898a00640 1 -- 172.21.15.43:0/3809455708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c037070 con 0x7fa89c0d1620 2024-04-26T20:27:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa898a00640 1 -- 172.21.15.43:0/3809455708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c032440 con 0x7fa89c0d1620 2024-04-26T20:27:45.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa898a00640 1 -- 172.21.15.43:0/3809455708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c032810 con 0x7fa89c0d1620 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 -- 172.21.15.43:0/3809455708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 msgr2=0x7fa89c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 --2- 172.21.15.43:0/3809455708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0d5750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009870 tx=0x7fa88c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 -- 172.21.15.43:0/3809455708 shutdown_connections 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 --2- 172.21.15.43:0/3809455708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0d1a00 0x7fa89c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 --2- 172.21.15.43:0/3809455708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0d5750 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 -- 172.21.15.43:0/3809455708 >> 172.21.15.43:0/3809455708 conn(0x7fa89c0d4c30 msgr2=0x7fa89c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 -- 172.21.15.43:0/3809455708 shutdown_connections 2024-04-26T20:27:45.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.717+0000 7fa89b200640 1 -- 172.21.15.43:0/3809455708 wait complete. 2024-04-26T20:27:45.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.719+0000 7fa89b200640 1 Processor -- start 2024-04-26T20:27:45.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.719+0000 7fa89b200640 1 -- start start 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0b50b0 0x7fa89c1e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa89c0d1a00 0x7fa89c14be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c0b5c50 con 0x7fa89c0d1620 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa89c0d1e60 con 0x7fa89c0d1a00 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa89b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa89c0b5dd0 con 0x7fa89c0b50b0 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.720+0000 7fa899400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52300/0 (socket says 172.21.15.43:52300) 2024-04-26T20:27:45.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 -- 172.21.15.43:0/2881517528 learned_addr learned my addr 172.21.15.43:0/2881517528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:45.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899e00640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0b50b0 0x7fa89c1e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa89a800640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa89c0d1a00 0x7fa89c14be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 -- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0b50b0 msgr2=0x7fa89c1e9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0b50b0 0x7fa89c1e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 -- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa89c0d1a00 msgr2=0x7fa89c14be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa89c0d1a00 0x7fa89c14be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 -- 172.21.15.43:0/2881517528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89c0cb450 con 0x7fa89c0d1620 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.721+0000 7fa899400640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ba50 tx=0x7fa88400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.722+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884019070 con 0x7fa89c0d1620 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.722+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88400cc00 con 0x7fa89c0d1620 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.722+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884014440 con 0x7fa89c0d1620 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.722+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89c0c9ff0 con 0x7fa89c0d1620 2024-04-26T20:27:45.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.722+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa89c0ca970 con 0x7fa89c0d1620 2024-04-26T20:27:45.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.723+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa89c069350 con 0x7fa89c0d1620 2024-04-26T20:27:45.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.724+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa88401c070 con 0x7fa89c0d1620 2024-04-26T20:27:45.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.725+0000 7fa892000640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:45.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.725+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fa88408b760 con 0x7fa89c0d1620 2024-04-26T20:27:45.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.725+0000 7fa899e00640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:45.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.726+0000 7fa899e00640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 0x7fa87806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa88c034000 tx=0x7fa88c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:45.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.729+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa89c0cb450 con 0x7fa89c0d1620 2024-04-26T20:27:45.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.937+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa89c0b1860 con 0x7fa89c0d1620 2024-04-26T20:27:45.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.940+0000 7fa892000640 1 -- 172.21.15.43:0/2881517528 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13140 (secure 0 0 0) 0x7fa884059cb0 con 0x7fa89c0d1620 2024-04-26T20:27:45.941 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:45.942 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:27:41.739040+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 msgr2=0x7fa87806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 0x7fa87806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa88c034000 tx=0x7fa88c02e000 comp rx=0 tx=0).stop 2024-04-26T20:27:45.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 msgr2=0x7fa89c0c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ba50 tx=0x7fa88400bf10 comp rx=0 tx=0).stop 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 shutdown_connections 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa87806c8f0 0x7fa87806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa89c0d1a00 0x7fa89c14be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89c0d1620 0x7fa89c0c9ab0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 --2- 172.21.15.43:0/2881517528 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa89c0b50b0 0x7fa89c1e9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.942+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 >> 172.21.15.43:0/2881517528 conn(0x7fa89c0d4c30 msgr2=0x7fa89c0d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.943+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 shutdown_connections 2024-04-26T20:27:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:45.943+0000 7fa89b200640 1 -- 172.21.15.43:0/2881517528 wait complete. 2024-04-26T20:27:45.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:27:45.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:27:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.172+0000 7ff867200640 1 Processor -- start 2024-04-26T20:27:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.172+0000 7ff867200640 1 -- start start 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff867200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff868155cd0 0x7ff868072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff867200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8680b5b10 0x7ff868162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff867200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86815f920 con 0x7ff868072b40 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff867200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff86815fc00 con 0x7ff868155cd0 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff867200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff86815fee0 con 0x7ff8680b5b10 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.174+0000 7ff865e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff868155cd0 0x7ff868072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff868155cd0 0x7ff868072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47182/0 (socket says 172.21.15.43:47182) 2024-04-26T20:27:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 -- 172.21.15.43:0/3445272581 learned_addr learned my addr 172.21.15.43:0/3445272581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:46.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 -- 172.21.15.43:0/3445272581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8680b5b10 msgr2=0x7ff868162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:46.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 --2- 172.21.15.43:0/3445272581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8680b5b10 0x7ff868162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 -- 172.21.15.43:0/3445272581 >> v1:172.21.15.43:6789/0 conn(0x7ff868072b40 legacy=0x7ff8680b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.175+0000 7ff865e00640 1 -- 172.21.15.43:0/3445272581 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681601c0 con 0x7ff868155cd0 2024-04-26T20:27:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.176+0000 7ff865e00640 1 --2- 172.21.15.43:0/3445272581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff868155cd0 0x7ff868072600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85802a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c42203101d3033 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.176+0000 7ff864a00640 1 -- 172.21.15.43:0/3445272581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858037070 con 0x7ff868155cd0 2024-04-26T20:27:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff864a00640 1 -- 172.21.15.43:0/3445272581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff858032640 con 0x7ff868155cd0 2024-04-26T20:27:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff864a00640 1 -- 172.21.15.43:0/3445272581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858032a10 con 0x7ff868155cd0 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 -- 172.21.15.43:0/3445272581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff868155cd0 msgr2=0x7ff868072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 --2- 172.21.15.43:0/3445272581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff868155cd0 0x7ff868072600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85802a320 comp rx=0 tx=0).stop 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 -- 172.21.15.43:0/3445272581 shutdown_connections 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 --2- 172.21.15.43:0/3445272581 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8680b5b10 0x7ff868162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 --2- 172.21.15.43:0/3445272581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff868155cd0 0x7ff868072600 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 -- 172.21.15.43:0/3445272581 >> 172.21.15.43:0/3445272581 conn(0x7ff868068090 msgr2=0x7ff8680684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 -- 172.21.15.43:0/3445272581 shutdown_connections 2024-04-26T20:27:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.177+0000 7ff867200640 1 -- 172.21.15.43:0/3445272581 wait complete. 2024-04-26T20:27:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.179+0000 7ff867200640 1 Processor -- start 2024-04-26T20:27:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.179+0000 7ff867200640 1 -- start start 2024-04-26T20:27:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff868072b40 0x7ff8680b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 0x7ff868150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff868156230 con 0x7ff8680b5b10 2024-04-26T20:27:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681560b0 con 0x7ff868155cd0 2024-04-26T20:27:46.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.180+0000 7ff867200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff86815f3f0 con 0x7ff868072b40 2024-04-26T20:27:46.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47196/0 (socket says 172.21.15.43:47196) 2024-04-26T20:27:46.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 -- 172.21.15.43:0/2753372290 learned_addr learned my addr 172.21.15.43:0/2753372290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865e00640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff868072b40 0x7ff8680b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff866800640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 0x7ff868150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 -- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff868072b40 msgr2=0x7ff8680b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff868072b40 0x7ff8680b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 -- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 msgr2=0x7ff868150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 0x7ff868150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff865400640 1 -- 172.21.15.43:0/2753372290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681539b0 con 0x7ff8680b5b10 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.181+0000 7ff866800640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 0x7ff868150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff865400640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c430 tx=0x7ff85400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540052d0 con 0x7ff8680b5b10 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff854005490 con 0x7ff8680b5b10 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854007260 con 0x7ff8680b5b10 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff868151be0 con 0x7ff8680b5b10 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.182+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681521a0 con 0x7ff8680b5b10 2024-04-26T20:27:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.183+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8681523a0 con 0x7ff8680b5b10 2024-04-26T20:27:46.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.184+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff854007400 con 0x7ff8680b5b10 2024-04-26T20:27:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.186+0000 7ff85e000640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 0x7ff84806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.186+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7ff85408af90 con 0x7ff8680b5b10 2024-04-26T20:27:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.186+0000 7ff865e00640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 0x7ff84806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.187+0000 7ff865e00640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 0x7ff84806ede0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff858034000 tx=0x7ff85802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.191+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8681523a0 con 0x7ff8680b5b10 2024-04-26T20:27:46.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.396+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8680506c0 con 0x7ff8680b5b10 2024-04-26T20:27:46.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.397+0000 7ff85e000640 1 -- 172.21.15.43:0/2753372290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7ff8540594e0 con 0x7ff8680b5b10 2024-04-26T20:27:46.398 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:27:46.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.399+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 msgr2=0x7ff84806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.399+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 0x7ff84806ede0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff858034000 tx=0x7ff85802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:46.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.399+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 msgr2=0x7ff8681526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.399+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c430 tx=0x7ff85400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 shutdown_connections 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff84806c920 0x7ff84806ede0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff868155cd0 0x7ff868150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8680b5b10 0x7ff8681526c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 --2- 172.21.15.43:0/2753372290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff868072b40 0x7ff8680b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 >> 172.21.15.43:0/2753372290 conn(0x7ff868068090 msgr2=0x7ff8680b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 shutdown_connections 2024-04-26T20:27:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.400+0000 7ff867200640 1 -- 172.21.15.43:0/2753372290 wait complete. 2024-04-26T20:27:46.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:27:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.634+0000 7f55a3e00640 1 Processor -- start 2024-04-26T20:27:46.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.635+0000 7f55a3e00640 1 -- start start 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.636+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c160ae0 0x7f559c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.636+0000 7f55a3e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559c161170 con 0x7f559c13fc90 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.636+0000 7f55a3e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559c161450 con 0x7f559c0d0da0 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.636+0000 7f55a3e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f559c161a10 con 0x7f559c160ae0 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.637+0000 7f55a1400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f559c13fc90 0x7f559c142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37724/0 (socket says 172.21.15.43:37724) 2024-04-26T20:27:46.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.637+0000 7f55a1400640 1 -- 172.21.15.43:0/1883149055 learned_addr learned my addr 172.21.15.43:0/1883149055 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:46.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.637+0000 7f55a0a00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c160ae0 0x7f559c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.637+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873019010 0 0) 0x7f559c161170 con 0x7f559c13fc90 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.637+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5584003520 con 0x7f559c13fc90 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182796643 0 0) 0x7f559c161450 con 0x7f559c0d0da0 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f559c161170 con 0x7f559c0d0da0 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341403748 0 0) 0x7f5584003520 con 0x7f559c13fc90 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c160ae0 msgr2=0x7f559c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c160ae0 0x7f559c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 >> v1:172.21.15.164:6789/0 conn(0x7f559c0d0da0 legacy=0x7f559c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c161730 con 0x7f559c13fc90 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f5590001670 con 0x7f559c13fc90 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 >> v1:172.21.15.43:6789/0 conn(0x7f559c13fc90 legacy=0x7f559c142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5584003b70 0x7f5584003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5584004800 0x7f5584004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55840051c0 0x7f558400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f558400bfb0 con 0x7f5584003b70 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f558400c130 con 0x7f55840051c0 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f558400c330 con 0x7f5584004800 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.638+0000 7f55a1400640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5584003b70 0x7f5584003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1e00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55840051c0 0x7f558400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a0a00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5584004800 0x7f5584004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1400640 1 -- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5584004800 msgr2=0x7f5584004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1400640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5584004800 0x7f5584004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1400640 1 -- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55840051c0 msgr2=0x7f558400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1400640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55840051c0 0x7f558400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1400640 1 -- 172.21.15.43:0/1883149055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584003520 con 0x7f5584003b70 2024-04-26T20:27:46.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.639+0000 7f55a1e00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55840051c0 0x7f558400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.640+0000 7f55a1400640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5584003b70 0x7f5584003f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f559000c190 tx=0x7f5584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba9324293a234fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007ce0 con 0x7f5584003b70 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590005ad0 con 0x7f5584003b70 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f559be00640 1 -- 172.21.15.43:0/1883149055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007070 con 0x7f5584003b70 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f55a3e00640 1 -- 172.21.15.43:0/1883149055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5584003b70 msgr2=0x7f5584003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1883149055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5584003b70 0x7f5584003f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f559000c190 tx=0x7f5584005830 comp rx=0 tx=0).stop 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f55a1e00640 1 -- 172.21.15.43:0/1883149055 reap_dead start 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f55a3e00640 1 -- 172.21.15.43:0/1883149055 shutdown_connections 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.641+0000 7f55a3e00640 1 -- 172.21.15.43:0/1883149055 >> 172.21.15.43:0/1883149055 conn(0x7f559c0682a0 msgr2=0x7f559c0686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.642+0000 7f55a3e00640 1 -- 172.21.15.43:0/1883149055 shutdown_connections 2024-04-26T20:27:46.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.642+0000 7f55a3e00640 1 -- 172.21.15.43:0/1883149055 wait complete. 2024-04-26T20:27:46.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.643+0000 7f55a3e00640 1 Processor -- start 2024-04-26T20:27:46.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.643+0000 7f55a3e00640 1 -- start start 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 0x7f559c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c155f20 0x7f559c1f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0d08c0 con 0x7f559c0cc7c0 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f559c160f20 con 0x7f5584003b70 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.644+0000 7f55a3e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f559c1433a0 con 0x7f559c155f20 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47210/0 (socket says 172.21.15.43:47210) 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 -- 172.21.15.43:0/1859877265 learned_addr learned my addr 172.21.15.43:0/1859877265 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a1400640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 0x7f559c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 -- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c155f20 msgr2=0x7f559c1f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c155f20 0x7f559c1f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 -- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 msgr2=0x7f559c0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 0x7f559c0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 -- 172.21.15.43:0/1859877265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c158190 con 0x7f559c0cc7c0 2024-04-26T20:27:46.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a1400640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 0x7f559c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:46.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.645+0000 7f55a0a00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5588011800 tx=0x7f5588011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.646+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558800f940 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.646+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f558800fae0 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.646+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588015460 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.646+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c158470 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.646+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c158a30 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.647+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c158c70 con 0x7f559c0cc7c0 2024-04-26T20:27:46.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.648+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5588015600 con 0x7f559c0cc7c0 2024-04-26T20:27:46.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.649+0000 7f5599600640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 0x7f557c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:46.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.649+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f558808c2a0 con 0x7f559c0cc7c0 2024-04-26T20:27:46.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.649+0000 7f55a1400640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 0x7f557c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:46.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.650+0000 7f55a1400640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 0x7f557c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5590032000 tx=0x7f5590009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:46.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.653+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5588019030 con 0x7f559c0cc7c0 2024-04-26T20:27:46.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.859+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f559c069350 con 0x7f559c0cc7c0 2024-04-26T20:27:46.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.860+0000 7f5599600640 1 -- 172.21.15.43:0/1859877265 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f558805a880 con 0x7f559c0cc7c0 2024-04-26T20:27:46.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:27:46.862 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-26T20:27:46.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.862+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 msgr2=0x7f557c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.862+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 0x7f557c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5590032000 tx=0x7f5590009030 comp rx=0 tx=0).stop 2024-04-26T20:27:46.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.862+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 msgr2=0x7f559c1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:46.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.862+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5588011800 tx=0x7f5588011cc0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 shutdown_connections 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f557c06c8f0 0x7f557c06edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f559c155f20 0x7f559c1f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f559c0cc7c0 0x7f559c1559e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 --2- 172.21.15.43:0/1859877265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5584003b70 0x7f559c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 >> 172.21.15.43:0/1859877265 conn(0x7f559c0682a0 msgr2=0x7f559c140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 shutdown_connections 2024-04-26T20:27:46.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:46.863+0000 7f55a3e00640 1 -- 172.21.15.43:0/1859877265 wait complete. 2024-04-26T20:27:46.884 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:27:46.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:27:47.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.099+0000 7efecca00640 1 Processor -- start 2024-04-26T20:27:47.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.099+0000 7efecca00640 1 -- start start 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efecca00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 0x7efec80ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efecca00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efec80b3ab0 0x7efec80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efecca00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec815d260 con 0x7efec80b00a0 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efecca00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7efec80ce600 con 0x7efec80b3ab0 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efecca00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efec815d540 con 0x7efec80cdc80 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efec7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efec80b3ab0 0x7efec80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efec7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efec80b3ab0 0x7efec80b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47226/0 (socket says 172.21.15.43:47226) 2024-04-26T20:27:47.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.101+0000 7efec7200640 1 -- 172.21.15.43:0/700485356 learned_addr learned my addr 172.21.15.43:0/700485356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec5e00640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 0x7efec80ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec7200640 1 -- 172.21.15.43:0/700485356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 msgr2=0x7efec80ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec7200640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 0x7efec80ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec7200640 1 -- 172.21.15.43:0/700485356 >> v1:172.21.15.164:6789/0 conn(0x7efec80b00a0 legacy=0x7efec80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec7200640 1 -- 172.21.15.43:0/700485356 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec815d820 con 0x7efec80b3ab0 2024-04-26T20:27:47.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.102+0000 7efec5e00640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 0x7efec80ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.103+0000 7efec7200640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80b3ab0 0x7efec80b3f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e50 tx=0x7efec002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db6d63b867cbc9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.104+0000 7efec5400640 1 -- 172.21.15.43:0/700485356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0036070 con 0x7efec80b3ab0 2024-04-26T20:27:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.104+0000 7efec5400640 1 -- 172.21.15.43:0/700485356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0031600 con 0x7efec80b3ab0 2024-04-26T20:27:47.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.104+0000 7efec5400640 1 -- 172.21.15.43:0/700485356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00319d0 con 0x7efec80b3ab0 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.104+0000 7efecca00640 1 -- 172.21.15.43:0/700485356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80b3ab0 msgr2=0x7efec80b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.104+0000 7efecca00640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80b3ab0 0x7efec80b3f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e50 tx=0x7efec002a6d0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 -- 172.21.15.43:0/700485356 shutdown_connections 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80b3ab0 0x7efec80b3f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 --2- 172.21.15.43:0/700485356 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80cdc80 0x7efec80ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 -- 172.21.15.43:0/700485356 >> 172.21.15.43:0/700485356 conn(0x7efec80b26b0 msgr2=0x7efec80b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 -- 172.21.15.43:0/700485356 shutdown_connections 2024-04-26T20:27:47.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.105+0000 7efecca00640 1 -- 172.21.15.43:0/700485356 wait complete. 2024-04-26T20:27:47.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.106+0000 7efecca00640 1 Processor -- start 2024-04-26T20:27:47.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.107+0000 7efecca00640 1 -- start start 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.107+0000 7efecca00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 0x7efec80acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efecca00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec80b3ab0 0x7efec80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efecca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efecca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efec806f670 con 0x7efec80cdc80 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efecca00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efec80d0580 con 0x7efec80b3ab0 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efecca00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efec806f7f0 con 0x7efec80b00a0 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47242/0 (socket says 172.21.15.43:47242) 2024-04-26T20:27:47.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 -- 172.21.15.43:0/3344124523 learned_addr learned my addr 172.21.15.43:0/3344124523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec6800640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 0x7efec80acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec5e00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec80b3ab0 0x7efec80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 -- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 msgr2=0x7efec80acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 0x7efec80acde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 -- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec80b3ab0 msgr2=0x7efec80aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec80b3ab0 0x7efec80aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec7200640 1 -- 172.21.15.43:0/3344124523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec8142d90 con 0x7efec80cdc80 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.108+0000 7efec6800640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 0x7efec80acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efec7200640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e50 tx=0x7efec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0036050 con 0x7efec80cdc80 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0042070 con 0x7efec80cdc80 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0009510 con 0x7efec80cdc80 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec8143030 con 0x7efec80cdc80 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.109+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec81435f0 con 0x7efec80cdc80 2024-04-26T20:27:47.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.110+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec8069350 con 0x7efec80cdc80 2024-04-26T20:27:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.111+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efec0007bd0 con 0x7efec80cdc80 2024-04-26T20:27:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.112+0000 7efebea00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 0x7efea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.112+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7efec00ac960 con 0x7efec80cdc80 2024-04-26T20:27:47.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.113+0000 7efec6800640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 0x7efea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:47.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.113+0000 7efec6800640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 0x7efea806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efeb8003eb0 tx=0x7efeb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:47.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.117+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efec0045360 con 0x7efec80cdc80 2024-04-26T20:27:47.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.319+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efec80ccdc0 con 0x7efec80cdc80 2024-04-26T20:27:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.848+0000 7efebea00640 1 -- 172.21.15.43:0/3344124523 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7efec007af40 con 0x7efec80cdc80 2024-04-26T20:27:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-26T20:27:47.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.852+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 msgr2=0x7efea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:47.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.852+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 0x7efea806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efeb8003eb0 tx=0x7efeb8009040 comp rx=0 tx=0).stop 2024-04-26T20:27:47.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.852+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 msgr2=0x7efec812fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:47.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.852+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efec0009e50 tx=0x7efec0004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 shutdown_connections 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7efea806c8f0 0x7efea806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efec80cdc80 0x7efec812fed0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efec80b3ab0 0x7efec80aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 --2- 172.21.15.43:0/3344124523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efec80b00a0 0x7efec80acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 >> 172.21.15.43:0/3344124523 conn(0x7efec80b26b0 msgr2=0x7efec8145830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 shutdown_connections 2024-04-26T20:27:47.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:47.853+0000 7efecca00640 1 -- 172.21.15.43:0/3344124523 wait complete. 2024-04-26T20:27:47.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:27:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.080+0000 7ff769600640 1 Processor -- start 2024-04-26T20:27:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.081+0000 7ff769600640 1 -- start start 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.082+0000 7ff769600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7640d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.082+0000 7ff769600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff7640b4390 0x7ff7640d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.082+0000 7ff769600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7640d3020 con 0x7ff7640d3980 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.082+0000 7ff769600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff7640b47c0 con 0x7ff7640b4390 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.082+0000 7ff769600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff7640b5060 con 0x7ff76415f310 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff763400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7640d3980 0x7ff7640cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37740/0 (socket says 172.21.15.43:37740) 2024-04-26T20:27:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff763400640 1 -- 172.21.15.43:0/3586096429 learned_addr learned my addr 172.21.15.43:0/3586096429 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff762000640 1 --2- 172.21.15.43:0/3586096429 >> v2:172.21.15.164:3300/0 conn(0x7ff7640b4390 0x7ff7640d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff762a00640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7640d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3269836090 0 0) 0x7ff7640d3020 con 0x7ff7640d3980 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.083+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff744003520 con 0x7ff7640d3980 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2854465339 0 0) 0x7ff744003520 con 0x7ff7640d3980 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 msgr2=0x7ff7640d5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7640d5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 >> v2:172.21.15.164:3300/0 conn(0x7ff7640b4390 msgr2=0x7ff7640d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 --2- 172.21.15.43:0/3586096429 >> v2:172.21.15.164:3300/0 conn(0x7ff7640b4390 0x7ff7640d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640b4a10 con 0x7ff7640d3980 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7ff75c001670 con 0x7ff7640d3980 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 >> v1:172.21.15.43:6789/0 conn(0x7ff7640d3980 legacy=0x7ff7640cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744003f80 0x7ff744004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff744004d40 0x7ff7440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640b47c0 con 0x7ff744004d40 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640b5060 con 0x7ff744003f80 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.084+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff74400ba70 con 0x7ff76415f310 2024-04-26T20:27:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff744004d40 0x7ff7440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff762a00640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff762000640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744003f80 0x7ff744004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 -- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 msgr2=0x7ff7440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff76415f310 0x7ff7440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 -- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744003f80 msgr2=0x7ff744004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744003f80 0x7ff744004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.085+0000 7ff763400640 1 -- 172.21.15.43:0/3586096429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff744003520 con 0x7ff744004d40 2024-04-26T20:27:48.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.086+0000 7ff763400640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff744004d40 0x7ff7440051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c190 tx=0x7ff744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef04ed0a9ce8c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007c80 con 0x7ff744004d40 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c005ad0 con 0x7ff744004d40 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff761600640 1 -- 172.21.15.43:0/3586096429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007070 con 0x7ff744004d40 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff769600640 1 -- 172.21.15.43:0/3586096429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff744004d40 msgr2=0x7ff7440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff769600640 1 --2- 172.21.15.43:0/3586096429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff744004d40 0x7ff7440051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c190 tx=0x7ff744005830 comp rx=0 tx=0).stop 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff763400640 1 -- 172.21.15.43:0/3586096429 reap_dead start 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff769600640 1 -- 172.21.15.43:0/3586096429 shutdown_connections 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff769600640 1 -- 172.21.15.43:0/3586096429 >> 172.21.15.43:0/3586096429 conn(0x7ff7640d4e60 msgr2=0x7ff7640d5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.087+0000 7ff769600640 1 -- 172.21.15.43:0/3586096429 shutdown_connections 2024-04-26T20:27:48.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.088+0000 7ff769600640 1 -- 172.21.15.43:0/3586096429 wait complete. 2024-04-26T20:27:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.089+0000 7ff769600640 1 Processor -- start 2024-04-26T20:27:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.089+0000 7ff769600640 1 -- start start 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744004d40 0x7ff764160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff764150180 0x7ff7640cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640d2dc0 con 0x7ff7641623c0 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640d2c40 con 0x7ff744004d40 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.090+0000 7ff769600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff7640b6500 con 0x7ff764150180 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47248/0 (socket says 172.21.15.43:47248) 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 -- 172.21.15.43:0/3510237707 learned_addr learned my addr 172.21.15.43:0/3510237707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 -- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff764150180 msgr2=0x7ff7640cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff764150180 0x7ff7640cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 -- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744004d40 msgr2=0x7ff764160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744004d40 0x7ff764160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 -- 172.21.15.43:0/3510237707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff764161320 con 0x7ff7641623c0 2024-04-26T20:27:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.091+0000 7ff762000640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f50 tx=0x7ff74c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.092+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c01a030 con 0x7ff7641623c0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.092+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74c011e80 con 0x7ff7641623c0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.092+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff764161600 con 0x7ff7641623c0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.092+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c003990 con 0x7ff7641623c0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.092+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff764161bc0 con 0x7ff7641623c0 2024-04-26T20:27:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.093+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff764069350 con 0x7ff7641623c0 2024-04-26T20:27:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.094+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff74c01d070 con 0x7ff7641623c0 2024-04-26T20:27:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.095+0000 7ff75aa00640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.095+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7ff74c08c550 con 0x7ff7641623c0 2024-04-26T20:27:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.096+0000 7ff762a00640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.096+0000 7ff762a00640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 0x7ff74006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff754009910 tx=0x7ff754008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.101+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff74c01d360 con 0x7ff7641623c0 2024-04-26T20:27:48.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.305+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff7640d3700 con 0x7ff7641623c0 2024-04-26T20:27:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.855+0000 7ff75aa00640 1 -- 172.21.15.43:0/3510237707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7ff74c05a9d0 con 0x7ff7641623c0 2024-04-26T20:27:48.857 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-26T20:27:48.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.859+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 msgr2=0x7ff74006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.859+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 0x7ff74006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff754009910 tx=0x7ff754008040 comp rx=0 tx=0).stop 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.859+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 msgr2=0x7ff7640c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.859+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f50 tx=0x7ff74c0115a0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 shutdown_connections 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff764150180 0x7ff7640cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7641623c0 0x7ff7640c8d50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 --2- 172.21.15.43:0/3510237707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff744004d40 0x7ff764160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.860+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 >> 172.21.15.43:0/3510237707 conn(0x7ff7640d4e60 msgr2=0x7ff7640d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.861+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 shutdown_connections 2024-04-26T20:27:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:48.861+0000 7ff769600640 1 -- 172.21.15.43:0/3510237707 wait complete. 2024-04-26T20:27:48.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:27:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.074+0000 7f5727e00640 1 Processor -- start 2024-04-26T20:27:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.074+0000 7f5727e00640 1 -- start start 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.075+0000 7f5727e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 0x7f57280b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.075+0000 7f5727e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57280b3900 con 0x7f572815a7e0 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.075+0000 7f5727e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57280b3b70 con 0x7f57280b5410 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.075+0000 7f5727e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f572815abc0 con 0x7f572815d9c0 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.076+0000 7f5726a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f572815a7e0 0x7f57280b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37742/0 (socket says 172.21.15.43:37742) 2024-04-26T20:27:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.076+0000 7f5726a00640 1 -- 172.21.15.43:0/874719478 learned_addr learned my addr 172.21.15.43:0/874719478 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.076+0000 7f5727400640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 0x7f57280b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.076+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818772310 0 0) 0x7f57280b3900 con 0x7f572815a7e0 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.076+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f570c003520 con 0x7f572815a7e0 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2313904254 0 0) 0x7f570c003520 con 0x7f572815a7e0 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 msgr2=0x7f57280b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 0x7f57280b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 >> v1:172.21.15.164:6789/0 conn(0x7f57280b5410 legacy=0x7f572815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57280b3e50 con 0x7f572815a7e0 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f5718001670 con 0x7f572815a7e0 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 >> v1:172.21.15.43:6789/0 conn(0x7f572815a7e0 legacy=0x7f57280b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f570c003a50 0x7f570c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 0x7f570c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c004d40 0x7f570c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f572815abc0 con 0x7f570c003a50 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f570c00bb50 con 0x7f570c004d40 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f570c00bd20 con 0x7f572815d9c0 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.077+0000 7f5726a00640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f570c003a50 0x7f570c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.078+0000 7f5726a00640 1 -- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 msgr2=0x7f570c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.078+0000 7f5726a00640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f572815d9c0 0x7f570c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.078+0000 7f5726a00640 1 -- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c004d40 msgr2=0x7f570c0051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.078+0000 7f5726a00640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c004d40 0x7f570c0051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.078+0000 7f5726a00640 1 -- 172.21.15.43:0/874719478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c003520 con 0x7f570c003a50 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.079+0000 7f5726a00640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f570c003a50 0x7f570c003e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f571800c110 tx=0x7f570c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1da0c15fd5d6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.079+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718007c40 con 0x7f570c003a50 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.079+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5718005af0 con 0x7f570c003a50 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.079+0000 7f5725600640 1 -- 172.21.15.43:0/874719478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718007070 con 0x7f570c003a50 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 -- 172.21.15.43:0/874719478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f570c003a50 msgr2=0x7f570c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 --2- 172.21.15.43:0/874719478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f570c003a50 0x7f570c003e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f571800c110 tx=0x7f570c005830 comp rx=0 tx=0).stop 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727400640 1 -- 172.21.15.43:0/874719478 reap_dead start 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 -- 172.21.15.43:0/874719478 shutdown_connections 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 -- 172.21.15.43:0/874719478 >> 172.21.15.43:0/874719478 conn(0x7f5728068090 msgr2=0x7f57280684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 -- 172.21.15.43:0/874719478 shutdown_connections 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.080+0000 7f5727e00640 1 -- 172.21.15.43:0/874719478 wait complete. 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.082+0000 7f5727e00640 1 Processor -- start 2024-04-26T20:27:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.082+0000 7f5727e00640 1 -- start start 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c003a50 0x7f57281549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5728145a50 0x7f57280ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f57280b5890 con 0x7f57281455d0 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f572815e010 con 0x7f570c003a50 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5727e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f57280b5a10 con 0x7f5728145a50 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47264/0 (socket says 172.21.15.43:47264) 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 -- 172.21.15.43:0/3893744703 learned_addr learned my addr 172.21.15.43:0/3893744703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:49.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 -- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5728145a50 msgr2=0x7f57280ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5728145a50 0x7f57280ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 -- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c003a50 msgr2=0x7f57281549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c003a50 0x7f57281549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.083+0000 7f5726000640 1 -- 172.21.15.43:0/3893744703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5728148f50 con 0x7f57281455d0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f5726000640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f571000c500 tx=0x7f571000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100052d0 con 0x7f57281455d0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710005470 con 0x7f57281455d0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5728154ee0 con 0x7f57281455d0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710012850 con 0x7f57281455d0 2024-04-26T20:27:49.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.084+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5728148990 con 0x7f57281455d0 2024-04-26T20:27:49.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.085+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572806f2a0 con 0x7f57281455d0 2024-04-26T20:27:49.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.087+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57100129f0 con 0x7f57281455d0 2024-04-26T20:27:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.087+0000 7f571ea00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 0x7f570406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.088+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f571008b970 con 0x7f57281455d0 2024-04-26T20:27:49.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.088+0000 7f5726a00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 0x7f570406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:49.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.089+0000 7f5726a00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 0x7f570406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5718032000 tx=0x7f5718009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:49.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.093+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f572806f2a0 con 0x7f57281455d0 2024-04-26T20:27:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.340+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f57280506c0 con 0x7f57281455d0 2024-04-26T20:27:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.880+0000 7f571ea00640 1 -- 172.21.15.43:0/3893744703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5710059c60 con 0x7f57281455d0 2024-04-26T20:27:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T20:27:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 msgr2=0x7f570406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 0x7f570406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5718032000 tx=0x7f5718009030 comp rx=0 tx=0).stop 2024-04-26T20:27:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 msgr2=0x7f5728152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f571000c500 tx=0x7f571000c9c0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 shutdown_connections 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f570406c8f0 0x7f570406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5728145a50 0x7f57280ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57281455d0 0x7f5728152d50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 --2- 172.21.15.43:0/3893744703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f570c003a50 0x7f57281549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.884+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 >> 172.21.15.43:0/3893744703 conn(0x7f5728068090 msgr2=0x7f57281562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.885+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 shutdown_connections 2024-04-26T20:27:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:49.886+0000 7f5727e00640 1 -- 172.21.15.43:0/3893744703 wait complete. 2024-04-26T20:27:49.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.096+0000 7faf23000640 1 Processor -- start 2024-04-26T20:27:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.096+0000 7faf23000640 1 -- start start 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.098+0000 7faf23000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 0x7faf1c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.098+0000 7faf23000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c0b0650 con 0x7faf1c15cab0 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.098+0000 7faf23000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c163a30 con 0x7faf1c0d0b40 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.098+0000 7faf23000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0d0900 con 0x7faf1c0d0f80 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1be00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faf1c15cab0 0x7faf1c0afae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37756/0 (socket says 172.21.15.43:37756) 2024-04-26T20:27:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1be00640 1 -- 172.21.15.43:0/4156514967 learned_addr learned my addr 172.21.15.43:0/4156514967 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277665875 0 0) 0x7faf1c0b0650 con 0x7faf1c15cab0 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf00003520 con 0x7faf1c15cab0 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf21000640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 0x7faf1c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1730078590 0 0) 0x7faf00003520 con 0x7faf1c15cab0 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 msgr2=0x7faf1c15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.099+0000 7faf1b400640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 0x7faf1c15fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 >> v1:172.21.15.164:6789/0 conn(0x7faf1c0d0b40 legacy=0x7faf1c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c162c70 con 0x7faf1c15cab0 2024-04-26T20:27:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7faf0c001670 con 0x7faf1c15cab0 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 >> v1:172.21.15.43:6789/0 conn(0x7faf1c15cab0 legacy=0x7faf1c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 0x7faf00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf0000bac0 con 0x7faf000039e0 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf0000bc70 con 0x7faf00004800 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf0000be70 con 0x7faf1c0d0f80 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf000039e0 0x7faf00003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 -- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 msgr2=0x7faf00005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf1c0d0f80 0x7faf00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 -- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf00004800 msgr2=0x7faf00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf00004800 0x7faf00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.100+0000 7faf1be00640 1 -- 172.21.15.43:0/4156514967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf00003520 con 0x7faf000039e0 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.101+0000 7faf1be00640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf000039e0 0x7faf00003e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c240 tx=0x7faf00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3495911fbcb4e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007d10 con 0x7faf000039e0 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c005ad0 con 0x7faf000039e0 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf1b400640 1 -- 172.21.15.43:0/4156514967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007070 con 0x7faf000039e0 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 -- 172.21.15.43:0/4156514967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf000039e0 msgr2=0x7faf00003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 --2- 172.21.15.43:0/4156514967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf000039e0 0x7faf00003e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c240 tx=0x7faf00005830 comp rx=0 tx=0).stop 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf21000640 1 -- 172.21.15.43:0/4156514967 reap_dead start 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 -- 172.21.15.43:0/4156514967 shutdown_connections 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 -- 172.21.15.43:0/4156514967 >> 172.21.15.43:0/4156514967 conn(0x7faf1c0b2900 msgr2=0x7faf1c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 -- 172.21.15.43:0/4156514967 shutdown_connections 2024-04-26T20:27:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.102+0000 7faf23000640 1 -- 172.21.15.43:0/4156514967 wait complete. 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.104+0000 7faf23000640 1 Processor -- start 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.104+0000 7faf23000640 1 -- start start 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf000039e0 0x7faf1c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 0x7faf1c14d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c163230 con 0x7faf1c072a20 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1635a0 con 0x7faf1c14d2e0 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf23000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0d13f0 con 0x7faf000039e0 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf13e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf13e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47280/0 (socket says 172.21.15.43:47280) 2024-04-26T20:27:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.105+0000 7faf13e00640 1 -- 172.21.15.43:0/1249184667 learned_addr learned my addr 172.21.15.43:0/1249184667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf1be00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf000039e0 0x7faf1c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf21000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 0x7faf1c14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 -- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf000039e0 msgr2=0x7faf1c1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf000039e0 0x7faf1c1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 -- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 msgr2=0x7faf1c14d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 0x7faf1c14d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 -- 172.21.15.43:0/1249184667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c0735c0 con 0x7faf1c072a20 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf21000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 0x7faf1c14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf13e00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c430 tx=0x7faf0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.106+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04017070 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.107+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf040052f0 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.107+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c0738a0 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.107+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04003830 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.107+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c14e5a0 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.108+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1c14e0e0 con 0x7faf1c072a20 2024-04-26T20:27:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.109+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf0401a070 con 0x7faf1c072a20 2024-04-26T20:27:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.110+0000 7faf18c00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 0x7faef806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.110+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7faf0408b0b0 con 0x7faf1c072a20 2024-04-26T20:27:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.110+0000 7faf1be00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 0x7faef806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.111+0000 7faf1be00640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 0x7faef806ede0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.115+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf1c14e0e0 con 0x7faf1c072a20 2024-04-26T20:27:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.313+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf1c069350 con 0x7faf1c072a20 2024-04-26T20:27:50.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.316+0000 7faf18c00640 1 -- 172.21.15.43:0/1249184667 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16931 (secure 0 0 0) 0x7faf040592c0 con 0x7faf1c072a20 2024-04-26T20:27:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:50.319 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":37,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:27:49.342267+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 msgr2=0x7faef806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 0x7faef806ede0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c02e000 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 msgr2=0x7faf1c072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c430 tx=0x7faf0400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 shutdown_connections 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faef806c920 0x7faef806ede0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf1c14d2e0 0x7faf1c14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf1c072a20 0x7faf1c072e80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 --2- 172.21.15.43:0/1249184667 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf000039e0 0x7faf1c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 >> 172.21.15.43:0/1249184667 conn(0x7faf1c0b2900 msgr2=0x7faf1c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 shutdown_connections 2024-04-26T20:27:50.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.319+0000 7faf23000640 1 -- 172.21.15.43:0/1249184667 wait complete. 2024-04-26T20:27:50.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:50.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.541+0000 7f4120c00640 1 Processor -- start 2024-04-26T20:27:50.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.542+0000 7f4120c00640 1 -- start start 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.543+0000 7f4120c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f411c0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.543+0000 7f4120c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c14d290 con 0x7f411c14ceb0 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4120c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0b5140 con 0x7f411c06f000 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4120c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0b5420 con 0x7f411c0cfe60 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4119e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f411c14ceb0 0x7f411c0cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37768/0 (socket says 172.21.15.43:37768) 2024-04-26T20:27:50.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4119e00640 1 -- 172.21.15.43:0/1753623783 learned_addr learned my addr 172.21.15.43:0/1753623783 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175107710 0 0) 0x7f411c14d290 con 0x7f411c14ceb0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4119400640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f411c0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.544+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4100003520 con 0x7f411c14ceb0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4074542886 0 0) 0x7f4100003520 con 0x7f411c14ceb0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 msgr2=0x7f411c0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f411c0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 >> v1:172.21.15.164:6789/0 conn(0x7f411c06f000 legacy=0x7f411c16d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0b5700 con 0x7f411c14ceb0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f410c001670 con 0x7f411c14ceb0 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 >> v1:172.21.15.43:6789/0 conn(0x7f411c14ceb0 legacy=0x7f411c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41000039e0 0x7f4100003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f4100004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0b5420 con 0x7f4100004d40 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f410000bbd0 con 0x7f41000039e0 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f410000bdd0 con 0x7f411c0cfe60 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.545+0000 7f411a800640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4100004d40 0x7f41000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f4119e00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41000039e0 0x7f4100003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f4119400640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f4100004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f411a800640 1 -- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 msgr2=0x7f4100004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f411a800640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f411c0cfe60 0x7f4100004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f411a800640 1 -- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41000039e0 msgr2=0x7f4100003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f411a800640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41000039e0 0x7f4100003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.546+0000 7f411a800640 1 -- 172.21.15.43:0/1753623783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100003520 con 0x7f4100004d40 2024-04-26T20:27:50.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.547+0000 7f411a800640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4100004d40 0x7f41000051c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4114001e50 tx=0x7f4100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a2a873a760573 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.547+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114001a70 con 0x7f4100004d40 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.547+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4114003970 con 0x7f4100004d40 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.547+0000 7f4118a00640 1 -- 172.21.15.43:0/1753623783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114003d60 con 0x7f4100004d40 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 -- 172.21.15.43:0/1753623783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4100004d40 msgr2=0x7f41000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 --2- 172.21.15.43:0/1753623783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4100004d40 0x7f41000051c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4114001e50 tx=0x7f4100005830 comp rx=0 tx=0).stop 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f411a800640 1 -- 172.21.15.43:0/1753623783 reap_dead start 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 -- 172.21.15.43:0/1753623783 shutdown_connections 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 -- 172.21.15.43:0/1753623783 >> 172.21.15.43:0/1753623783 conn(0x7f411c0b2900 msgr2=0x7f411c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 -- 172.21.15.43:0/1753623783 shutdown_connections 2024-04-26T20:27:50.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.548+0000 7f4120c00640 1 -- 172.21.15.43:0/1753623783 wait complete. 2024-04-26T20:27:50.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.550+0000 7f4120c00640 1 Processor -- start 2024-04-26T20:27:50.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.550+0000 7f4120c00640 1 -- start start 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 0x7f411c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 0x7f411c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0b4a60 con 0x7f411c144070 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0b5cc0 con 0x7f411c1566e0 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4120c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0b4be0 con 0x7f4100004d40 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4119400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4119400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47296/0 (socket says 172.21.15.43:47296) 2024-04-26T20:27:50.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.551+0000 7f4119400640 1 -- 172.21.15.43:0/62182758 learned_addr learned my addr 172.21.15.43:0/62182758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119e00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 0x7f411c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f411a800640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 0x7f411c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 -- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 msgr2=0x7f411c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 0x7f411c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 -- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 msgr2=0x7f411c155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 0x7f411c155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 -- 172.21.15.43:0/62182758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0c6430 con 0x7f411c144070 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f411a800640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 0x7f411c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119e00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 0x7f411c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4119400640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f410400e7e0 tx=0x7f410400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4104017070 con 0x7f411c144070 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410400c6d0 con 0x7f411c144070 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.552+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c0cd3c0 con 0x7f411c144070 2024-04-26T20:27:50.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.553+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410400f040 con 0x7f411c144070 2024-04-26T20:27:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.553+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c153e50 con 0x7f411c144070 2024-04-26T20:27:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.554+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c069350 con 0x7f411c144070 2024-04-26T20:27:50.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.555+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f410401a070 con 0x7f411c144070 2024-04-26T20:27:50.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.556+0000 7f4112000640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.556+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f410408b9c0 con 0x7f411c144070 2024-04-26T20:27:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.556+0000 7f4119e00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.557+0000 7f4119e00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 0x7f40f806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f410c001bf0 tx=0x7f410c000f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.561+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f411c0c6430 con 0x7f411c144070 2024-04-26T20:27:50.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.760+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f411c073a20 con 0x7f411c144070 2024-04-26T20:27:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.762+0000 7f4112000640 1 -- 172.21.15.43:0/62182758 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16931 (secure 0 0 0) 0x7f4104059bc0 con 0x7f411c144070 2024-04-26T20:27:50.763 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:50.764 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":37,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:27:49.342267+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 msgr2=0x7f40f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 0x7f40f806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f410c001bf0 tx=0x7f410c000f10 comp rx=0 tx=0).stop 2024-04-26T20:27:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 msgr2=0x7f411c0c5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f410400e7e0 tx=0x7f410400eca0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 shutdown_connections 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f40f806c8f0 0x7f40f806edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f411c1566e0 0x7f411c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f411c144070 0x7f411c0c5b70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 --2- 172.21.15.43:0/62182758 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4100004d40 0x7f411c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 >> 172.21.15.43:0/62182758 conn(0x7f411c0b2900 msgr2=0x7f411c0cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.764+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 shutdown_connections 2024-04-26T20:27:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.765+0000 7f4120c00640 1 -- 172.21.15.43:0/62182758 wait complete. 2024-04-26T20:27:50.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:27:50.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.993+0000 7f9fda800640 1 Processor -- start 2024-04-26T20:27:50.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.993+0000 7f9fda800640 1 -- start start 2024-04-26T20:27:50.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fda800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fd406f3a0 0x7f9fd40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fda800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9fd415bd60 0x7f9fd40b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fda800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd415c2a0 con 0x7f9fd415b980 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fda800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd415d060 con 0x7f9fd406f3a0 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fda800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd40b5090 con 0x7f9fd415bd60 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fd8a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fd406f3a0 0x7f9fd40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fd8a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fd406f3a0 0x7f9fd40b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47300/0 (socket says 172.21.15.43:47300) 2024-04-26T20:27:50.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.995+0000 7f9fd8a00640 1 -- 172.21.15.43:0/783037374 learned_addr learned my addr 172.21.15.43:0/783037374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.996+0000 7f9fd9e00640 1 --2- 172.21.15.43:0/783037374 >> v2:172.21.15.164:3300/0 conn(0x7f9fd415bd60 0x7f9fd40b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.996+0000 7f9fd8a00640 1 -- 172.21.15.43:0/783037374 >> v2:172.21.15.164:3300/0 conn(0x7f9fd415bd60 msgr2=0x7f9fd40b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.996+0000 7f9fd8a00640 1 --2- 172.21.15.43:0/783037374 >> v2:172.21.15.164:3300/0 conn(0x7f9fd415bd60 0x7f9fd40b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.996+0000 7f9fd8a00640 1 -- 172.21.15.43:0/783037374 >> v1:172.21.15.164:6789/0 conn(0x7f9fd415b980 legacy=0x7f9fd406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.996+0000 7f9fd8a00640 1 -- 172.21.15.43:0/783037374 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd415c580 con 0x7f9fd406f3a0 2024-04-26T20:27:50.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.997+0000 7f9fd8a00640 1 --2- 172.21.15.43:0/783037374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd406f3a0 0x7f9fd40b4610 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4072bd0 tx=0x7f9fc002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d588afa5ab682382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:50.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.997+0000 7f9fd3e00640 1 -- 172.21.15.43:0/783037374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0037070 con 0x7f9fd406f3a0 2024-04-26T20:27:50.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.997+0000 7f9fd3e00640 1 -- 172.21.15.43:0/783037374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc0032440 con 0x7f9fd406f3a0 2024-04-26T20:27:50.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.997+0000 7f9fd3e00640 1 -- 172.21.15.43:0/783037374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0032810 con 0x7f9fd406f3a0 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 -- 172.21.15.43:0/783037374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd406f3a0 msgr2=0x7f9fd40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 --2- 172.21.15.43:0/783037374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd406f3a0 0x7f9fd40b4610 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4072bd0 tx=0x7f9fc002a230 comp rx=0 tx=0).stop 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 -- 172.21.15.43:0/783037374 shutdown_connections 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 --2- 172.21.15.43:0/783037374 >> v2:172.21.15.164:3300/0 conn(0x7f9fd415bd60 0x7f9fd40b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 --2- 172.21.15.43:0/783037374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd406f3a0 0x7f9fd40b4610 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 -- 172.21.15.43:0/783037374 >> 172.21.15.43:0/783037374 conn(0x7f9fd4068090 msgr2=0x7f9fd40684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 -- 172.21.15.43:0/783037374 shutdown_connections 2024-04-26T20:27:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:50.998+0000 7f9fda800640 1 -- 172.21.15.43:0/783037374 wait complete. 2024-04-26T20:27:51.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.000+0000 7f9fda800640 1 Processor -- start 2024-04-26T20:27:51.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.000+0000 7f9fda800640 1 -- start start 2024-04-26T20:27:51.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fd406f3a0 0x7f9fd41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd415b980 0x7f9fd40cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd40b52b0 con 0x7f9fd415bd60 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd415cb40 con 0x7f9fd415b980 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.001+0000 7f9fda800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd406db80 con 0x7f9fd406f3a0 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47310/0 (socket says 172.21.15.43:47310) 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 -- 172.21.15.43:0/3112033544 learned_addr learned my addr 172.21.15.43:0/3112033544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9400640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fd406f3a0 0x7f9fd41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd8a00640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd415b980 0x7f9fd40cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 -- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fd406f3a0 msgr2=0x7f9fd41545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fd406f3a0 0x7f9fd41545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 -- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd415b980 msgr2=0x7f9fd40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd415b980 0x7f9fd40cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 -- 172.21.15.43:0/3112033544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd40cfe60 con 0x7f9fd415bd60 2024-04-26T20:27:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.002+0000 7f9fd9e00640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00cc30 tx=0x7f9fcc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:51.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.003+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00ce40 con 0x7f9fd415bd60 2024-04-26T20:27:51.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.003+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc003990 con 0x7f9fd415bd60 2024-04-26T20:27:51.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.003+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc003d40 con 0x7f9fd415bd60 2024-04-26T20:27:51.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.003+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd40d0140 con 0x7f9fd415bd60 2024-04-26T20:27:51.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.003+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd40d0700 con 0x7f9fd415bd60 2024-04-26T20:27:51.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.004+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd40506c0 con 0x7f9fd415bd60 2024-04-26T20:27:51.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.005+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fcc003ee0 con 0x7f9fd415bd60 2024-04-26T20:27:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.006+0000 7f9fd1600640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 0x7f9fb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.006+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9fcc08c1c0 con 0x7f9fd415bd60 2024-04-26T20:27:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.006+0000 7f9fd9400640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 0x7f9fb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.007+0000 7f9fd9400640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 0x7f9fb406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8004ea0 tx=0x7f9fc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:51.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.010+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9fcc014030 con 0x7f9fd415bd60 2024-04-26T20:27:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.257+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9fd4074f70 con 0x7f9fd415bd60 2024-04-26T20:27:51.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.954+0000 7f9fd1600640 1 -- 172.21.15.43:0/3112033544 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9fcc05a460 con 0x7f9fd415bd60 2024-04-26T20:27:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.958+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 msgr2=0x7f9fb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.958+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 0x7f9fb406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8004ea0 tx=0x7f9fc8009040 comp rx=0 tx=0).stop 2024-04-26T20:27:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.958+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 msgr2=0x7f9fd40cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.958+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00cc30 tx=0x7f9fcc00a5c0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 shutdown_connections 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f9fb406c8f0 0x7f9fb406edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fd415bd60 0x7f9fd40cf800 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9fd415b980 0x7f9fd40cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 --2- 172.21.15.43:0/3112033544 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9fd406f3a0 0x7f9fd41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 >> 172.21.15.43:0/3112033544 conn(0x7f9fd4068090 msgr2=0x7f9fd415e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 shutdown_connections 2024-04-26T20:27:51.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:51.959+0000 7f9fda800640 1 -- 172.21.15.43:0/3112033544 wait complete. 2024-04-26T20:27:51.982 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:27:51.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:27:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.197+0000 7f909f200640 1 Processor -- start 2024-04-26T20:27:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.198+0000 7f909f200640 1 -- start start 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.199+0000 7f909f200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90a015f3f0 0x7f90a00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.199+0000 7f909f200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a015f920 con 0x7f90a00b5940 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.199+0000 7f909f200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a015fc00 con 0x7f90a0155cd0 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.199+0000 7f909f200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f90a015fee0 con 0x7f90a015f3f0 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90a015f3f0 0x7f90a00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90a015f3f0 0x7f90a00b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47316/0 (socket says 172.21.15.43:47316) 2024-04-26T20:27:52.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 -- 172.21.15.43:0/627370991 learned_addr learned my addr 172.21.15.43:0/627370991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:52.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 -- 172.21.15.43:0/627370991 >> v1:172.21.15.164:6789/0 conn(0x7f90a0155cd0 legacy=0x7f90a0072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:52.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 -- 172.21.15.43:0/627370991 >> v1:172.21.15.43:6789/0 conn(0x7f90a00b5940 legacy=0x7f90a0162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.200+0000 7f909d400640 1 -- 172.21.15.43:0/627370991 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a01601c0 con 0x7f90a015f3f0 2024-04-26T20:27:52.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.201+0000 7f909d400640 1 --2- 172.21.15.43:0/627370991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a015f3f0 0x7f90a00b5400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9088009870 tx=0x7f908802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcd09a8c9380f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:52.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.202+0000 7f909ca00640 1 -- 172.21.15.43:0/627370991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088037070 con 0x7f90a015f3f0 2024-04-26T20:27:52.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.202+0000 7f909ca00640 1 -- 172.21.15.43:0/627370991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9088032640 con 0x7f90a015f3f0 2024-04-26T20:27:52.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.202+0000 7f909ca00640 1 -- 172.21.15.43:0/627370991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088032a10 con 0x7f90a015f3f0 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.202+0000 7f909f200640 1 -- 172.21.15.43:0/627370991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a015f3f0 msgr2=0x7f90a00b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.202+0000 7f909f200640 1 --2- 172.21.15.43:0/627370991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a015f3f0 0x7f90a00b5400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9088009870 tx=0x7f908802a330 comp rx=0 tx=0).stop 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.203+0000 7f909f200640 1 -- 172.21.15.43:0/627370991 shutdown_connections 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.203+0000 7f909f200640 1 --2- 172.21.15.43:0/627370991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a015f3f0 0x7f90a00b5400 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.203+0000 7f909f200640 1 -- 172.21.15.43:0/627370991 >> 172.21.15.43:0/627370991 conn(0x7f90a0068090 msgr2=0x7f90a00684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.203+0000 7f909f200640 1 -- 172.21.15.43:0/627370991 shutdown_connections 2024-04-26T20:27:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.203+0000 7f909f200640 1 -- 172.21.15.43:0/627370991 wait complete. 2024-04-26T20:27:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.204+0000 7f909f200640 1 Processor -- start 2024-04-26T20:27:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.204+0000 7f909f200640 1 -- start start 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90a00b5940 0x7f90a015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a0155cd0 0x7f90a01464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01560b0 con 0x7f90a01f4ce0 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0072d10 con 0x7f90a0155cd0 2024-04-26T20:27:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.205+0000 7f909f200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f90a0156230 con 0x7f90a00b5940 2024-04-26T20:27:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47322/0 (socket says 172.21.15.43:47322) 2024-04-26T20:27:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 -- 172.21.15.43:0/2746978728 learned_addr learned my addr 172.21.15.43:0/2746978728 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909de00640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90a00b5940 0x7f90a015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 -- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90a00b5940 msgr2=0x7f90a015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909d400640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a0155cd0 0x7f90a01464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90a00b5940 0x7f90a015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 -- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a0155cd0 msgr2=0x7f90a01464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a0155cd0 0x7f90a01464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.206+0000 7f909e800640 1 -- 172.21.15.43:0/2746978728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a01f5580 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f909e800640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f909800c7d0 tx=0x7f909800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098003b70 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098003d30 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800aa90 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a01f5860 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.207+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01f8300 con 0x7f90a01f4ce0 2024-04-26T20:27:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.208+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00506c0 con 0x7f90a01f4ce0 2024-04-26T20:27:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.209+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f909800ac30 con 0x7f90a01f4ce0 2024-04-26T20:27:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.210+0000 7f9096000640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 0x7f907c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.210+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f909808bce0 con 0x7f90a01f4ce0 2024-04-26T20:27:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.211+0000 7f909de00640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 0x7f907c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.211+0000 7f909de00640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 0x7f907c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f909000a750 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.215+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9098059ee0 con 0x7f90a01f4ce0 2024-04-26T20:27:52.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.448+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f90a0068610 con 0x7f90a01f4ce0 2024-04-26T20:27:52.734 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-04-26T20:27:52.733+0000 7f8c26c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:27:52.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.958+0000 7f9096000640 1 -- 172.21.15.43:0/2746978728 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f90980597e0 con 0x7f90a01f4ce0 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.961+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 msgr2=0x7f907c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.961+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 0x7f907c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f909000a750 tx=0x7f9090008040 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.961+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 msgr2=0x7f90a01f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.961+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f909800c7d0 tx=0x7f909800cc90 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 shutdown_connections 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f907c06c920 0x7f907c06ede0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90a01f4ce0 0x7f90a01f8b00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a0155cd0 0x7f90a01464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 --2- 172.21.15.43:0/2746978728 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f90a00b5940 0x7f90a015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 >> 172.21.15.43:0/2746978728 conn(0x7f90a0068090 msgr2=0x7f90a006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 shutdown_connections 2024-04-26T20:27:52.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:52.962+0000 7f909f200640 1 -- 172.21.15.43:0/2746978728 wait complete. 2024-04-26T20:27:52.987 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:27:52.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:27:53.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.197+0000 7f7be5400640 1 Processor -- start 2024-04-26T20:27:53.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.197+0000 7f7be5400640 1 -- start start 2024-04-26T20:27:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7be5400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7be00b3490 0x7f7be006f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7be5400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be00cf060 con 0x7f7be006f560 2024-04-26T20:27:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7be5400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be00cf340 con 0x7f7be0145590 2024-04-26T20:27:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7be5400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7be00d0100 con 0x7f7be00b3490 2024-04-26T20:27:53.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7bdf200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7be006f560 0x7f7be0071eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37788/0 (socket says 172.21.15.43:37788) 2024-04-26T20:27:53.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.199+0000 7f7bdf200640 1 -- 172.21.15.43:0/2304911272 learned_addr learned my addr 172.21.15.43:0/2304911272 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> v2:172.21.15.164:3300/0 conn(0x7f7be00b3490 0x7f7be006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475415018 0 0) 0x7f7be00cf060 con 0x7f7be006f560 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bc4003520 con 0x7f7be006f560 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3580717952 0 0) 0x7f7bc4003520 con 0x7f7be006f560 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 >> v2:172.21.15.164:3300/0 conn(0x7f7be00b3490 msgr2=0x7f7be006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 --2- 172.21.15.43:0/2304911272 >> v2:172.21.15.164:3300/0 conn(0x7f7be00b3490 0x7f7be006f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 >> v1:172.21.15.164:6789/0 conn(0x7f7be0145590 legacy=0x7f7be00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be00cf620 con 0x7f7be006f560 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f7bd80019f0 con 0x7f7be006f560 2024-04-26T20:27:53.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> v2:172.21.15.164:3300/0 conn(0x7f7be00b3490 0x7f7be006f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 >> v1:172.21.15.43:6789/0 conn(0x7f7be006f560 legacy=0x7f7be0071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bc4003a00 0x7f7bc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7bc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bc4005210 0x7f7bc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00d0100 con 0x7f7bc4004800 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc400c160 con 0x7f7bc4005210 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc400e570 con 0x7f7bc4003a00 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.200+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7bc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bdf200640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bc4005210 0x7f7bc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bde800640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bc4003a00 0x7f7bc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bd5e00640 1 -- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bc4003a00 msgr2=0x7f7bc4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bc4003a00 0x7f7bc4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bd5e00640 1 -- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bc4005210 msgr2=0x7f7bc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bc4005210 0x7f7bc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bd5e00640 1 -- 172.21.15.43:0/2304911272 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc4003520 con 0x7f7bc4004800 2024-04-26T20:27:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bde800640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7bc4003a00 0x7f7bc4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bdf200640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7bc4005210 0x7f7bc400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.201+0000 7f7bdf200640 1 -- 172.21.15.43:0/2304911272 reap_dead start 2024-04-26T20:27:53.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.202+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7bc4004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7be00b38d0 tx=0x7f7bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842b1bb1c3a428ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.202+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800f040 con 0x7f7bc4004800 2024-04-26T20:27:53.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.202+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc802ac90 con 0x7f7bc4004800 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.202+0000 7f7bdde00640 1 -- 172.21.15.43:0/2304911272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8004250 con 0x7f7bc4004800 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 -- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 msgr2=0x7f7bc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7bc4004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7be00b38d0 tx=0x7f7bc4005830 comp rx=0 tx=0).stop 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 -- 172.21.15.43:0/2304911272 shutdown_connections 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 --2- 172.21.15.43:0/2304911272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7bc4004c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 -- 172.21.15.43:0/2304911272 >> 172.21.15.43:0/2304911272 conn(0x7f7be00b2900 msgr2=0x7f7be00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 -- 172.21.15.43:0/2304911272 shutdown_connections 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.203+0000 7f7be5400640 1 -- 172.21.15.43:0/2304911272 wait complete. 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.205+0000 7f7be5400640 1 Processor -- start 2024-04-26T20:27:53.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.205+0000 7f7be5400640 1 -- start start 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7be0131030 0x7f7be01e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7be012e360 0x7f7be00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00cfbe0 con 0x7f7bc4004800 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7be006f990 con 0x7f7be012e360 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7be5400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7be0145970 con 0x7f7be0131030 2024-04-26T20:27:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7bde800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7bde800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47340/0 (socket says 172.21.15.43:47340) 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7bde800640 1 -- 172.21.15.43:0/2145046963 learned_addr learned my addr 172.21.15.43:0/2145046963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.206+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7be0131030 0x7f7be01e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bdf200640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7be012e360 0x7f7be00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 -- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7be0131030 msgr2=0x7f7be01e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7be0131030 0x7f7be01e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 -- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7be012e360 msgr2=0x7f7be00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7be012e360 0x7f7be00b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 -- 172.21.15.43:0/2145046963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be0130190 con 0x7f7bc4004800 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bde800640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d500 tx=0x7f7bd000d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0019070 con 0x7f7bc4004800 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd0007030 con 0x7f7bc4004800 2024-04-26T20:27:53.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01e8380 con 0x7f7bc4004800 2024-04-26T20:27:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0007400 con 0x7f7bc4004800 2024-04-26T20:27:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.207+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be012ee90 con 0x7f7bc4004800 2024-04-26T20:27:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.208+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be0069350 con 0x7f7bc4004800 2024-04-26T20:27:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.209+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bd001c070 con 0x7f7bc4004800 2024-04-26T20:27:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.210+0000 7f7bd7400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 0x7f7bbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.210+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7bd008b3d0 con 0x7f7bc4004800 2024-04-26T20:27:53.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.211+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 0x7f7bbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.211+0000 7f7bd5e00640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 0x7f7bbc06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8033000 tx=0x7f7bc802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.215+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bd0059660 con 0x7f7bc4004800 2024-04-26T20:27:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.454+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7be0074f00 con 0x7f7bc4004800 2024-04-26T20:27:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.455+0000 7f7bd7400640 1 -- 172.21.15.43:0/2145046963 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7f7bd0058f60 con 0x7f7bc4004800 2024-04-26T20:27:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:53.457 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:27:53.457 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:27:53.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 msgr2=0x7f7bbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 0x7f7bbc06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8033000 tx=0x7f7bc802e000 comp rx=0 tx=0).stop 2024-04-26T20:27:53.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 msgr2=0x7f7be012fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d500 tx=0x7f7bd000d9c0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 shutdown_connections 2024-04-26T20:27:53.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f7bbc06c8f0 0x7f7bbc06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7be0131030 0x7f7be01e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7be012e360 0x7f7be00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 --2- 172.21.15.43:0/2145046963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bc4004800 0x7f7be012fc50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 >> 172.21.15.43:0/2145046963 conn(0x7f7be00b2900 msgr2=0x7f7be00ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:53.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 shutdown_connections 2024-04-26T20:27:53.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.458+0000 7f7be5400640 1 -- 172.21.15.43:0/2145046963 wait complete. 2024-04-26T20:27:53.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:27:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.682+0000 7faa13000640 1 Processor -- start 2024-04-26T20:27:53.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.682+0000 7faa13000640 1 -- start start 2024-04-26T20:27:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa13000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa0c0cd180 0x7faa0c072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa13000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa13000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0c0cd560 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa13000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7faa0c072d00 con 0x7faa0c0cd180 2024-04-26T20:27:53.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa13000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c072fe0 con 0x7faa0c15dc50 2024-04-26T20:27:53.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa11c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa0c0cd180 0x7faa0c072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa11c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa0c0cd180 0x7faa0c072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47356/0 (socket says 172.21.15.43:47356) 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.684+0000 7faa11c00640 1 -- 172.21.15.43:0/1008148543 learned_addr learned my addr 172.21.15.43:0/1008148543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.685+0000 7faa11200640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.685+0000 7faa11c00640 1 -- 172.21.15.43:0/1008148543 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 msgr2=0x7faa0c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.685+0000 7faa11c00640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.685+0000 7faa11c00640 1 -- 172.21.15.43:0/1008148543 >> v1:172.21.15.43:6789/0 conn(0x7faa0c0ce2b0 legacy=0x7faa0c1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.685+0000 7faa11c00640 1 -- 172.21.15.43:0/1008148543 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c0732c0 con 0x7faa0c0cd180 2024-04-26T20:27:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.686+0000 7faa11c00640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0cd180 0x7faa0c072620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4919de895e5c01d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.686+0000 7faa0be00640 1 -- 172.21.15.43:0/1008148543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00037070 con 0x7faa0c0cd180 2024-04-26T20:27:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.686+0000 7faa0be00640 1 -- 172.21.15.43:0/1008148543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa00032440 con 0x7faa0c0cd180 2024-04-26T20:27:53.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.686+0000 7faa0be00640 1 -- 172.21.15.43:0/1008148543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00032810 con 0x7faa0c0cd180 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 -- 172.21.15.43:0/1008148543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0cd180 msgr2=0x7faa0c072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0cd180 0x7faa0c072620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0002a250 comp rx=0 tx=0).stop 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 -- 172.21.15.43:0/1008148543 shutdown_connections 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 --2- 172.21.15.43:0/1008148543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0cd180 0x7faa0c072620 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 -- 172.21.15.43:0/1008148543 >> 172.21.15.43:0/1008148543 conn(0x7faa0c068090 msgr2=0x7faa0c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 -- 172.21.15.43:0/1008148543 shutdown_connections 2024-04-26T20:27:53.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.687+0000 7faa13000640 1 -- 172.21.15.43:0/1008148543 wait complete. 2024-04-26T20:27:53.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.689+0000 7faa13000640 1 Processor -- start 2024-04-26T20:27:53.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.689+0000 7faa13000640 1 -- start start 2024-04-26T20:27:53.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c0cd180 0x7faa0c0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c1f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c06ead0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c073da0 con 0x7faa0c0cd180 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa13000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa0c15e0d0 con 0x7faa0c15dc50 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa11200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa11200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47364/0 (socket says 172.21.15.43:47364) 2024-04-26T20:27:53.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.690+0000 7faa11200640 1 -- 172.21.15.43:0/3980404774 learned_addr learned my addr 172.21.15.43:0/3980404774 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:53.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 -- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 msgr2=0x7faa0c1f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c1f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 -- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c0cd180 msgr2=0x7faa0c0c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11c00640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c0cd180 0x7faa0c0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c0cd180 0x7faa0c0c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 -- 172.21.15.43:0/3980404774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1e70c0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa11200640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d5f0 tx=0x7fa9f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80125f0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f8019070 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80129d0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.691+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c1e73a0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.692+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c0c5fc0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.693+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa0c069350 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.694+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9f8012b70 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.694+0000 7faa09600640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:53.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.695+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa9f808c0d0 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.695+0000 7faa11c00640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:53.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.695+0000 7faa11c00640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa00034000 tx=0x7faa0002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.699+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9f805a360 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.907+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa0c0d0670 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.911+0000 7faa09600640 1 -- 172.21.15.43:0/3980404774 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16918 (secure 0 0 0) 0x7fa9f8059c60 con 0x7faa0c0ce2b0 2024-04-26T20:27:53.913 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:53.913 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:27:50.897762+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:27:53.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.913+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 msgr2=0x7fa9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.913+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa00034000 tx=0x7faa0002e000 comp rx=0 tx=0).stop 2024-04-26T20:27:53.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.913+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 msgr2=0x7faa0c1f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.913+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800d5f0 tx=0x7fa9f800dab0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 shutdown_connections 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa9ec06c8f0 0x7fa9ec06edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa0c15dc50 0x7faa0c1f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa0c0ce2b0 0x7faa0c1f86e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 --2- 172.21.15.43:0/3980404774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa0c0cd180 0x7faa0c0c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 >> 172.21.15.43:0/3980404774 conn(0x7faa0c068090 msgr2=0x7faa0c0af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 shutdown_connections 2024-04-26T20:27:53.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:53.914+0000 7faa13000640 1 -- 172.21.15.43:0/3980404774 wait complete. 2024-04-26T20:27:53.933 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:27:53.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:27:54.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.131+0000 7f1825400640 1 Processor -- start 2024-04-26T20:27:54.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.131+0000 7f1825400640 1 -- start start 2024-04-26T20:27:54.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.132+0000 7f1825400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f182006e430 0x7f18200d34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.132+0000 7f1825400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f182006d520 0x7f1820070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.132+0000 7f1825400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f182006fdf0 con 0x7f182015ef90 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.132+0000 7f1825400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f18200700f0 con 0x7f182006e430 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.132+0000 7f1825400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f18200d17b0 con 0x7f182006d520 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.133+0000 7f181f400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f182006e430 0x7f18200d34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.133+0000 7f181f400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f182006e430 0x7f18200d34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47370/0 (socket says 172.21.15.43:47370) 2024-04-26T20:27:54.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.133+0000 7f181f400640 1 -- 172.21.15.43:0/2811616387 learned_addr learned my addr 172.21.15.43:0/2811616387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.133+0000 7f1824a00640 1 --2- 172.21.15.43:0/2811616387 >> v2:172.21.15.164:3300/0 conn(0x7f182006d520 0x7f1820070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.133+0000 7f181f400640 1 -- 172.21.15.43:0/2811616387 >> v2:172.21.15.164:3300/0 conn(0x7f182006d520 msgr2=0x7f1820070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:54.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.134+0000 7f181f400640 1 --2- 172.21.15.43:0/2811616387 >> v2:172.21.15.164:3300/0 conn(0x7f182006d520 0x7f1820070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.134+0000 7f181f400640 1 -- 172.21.15.43:0/2811616387 >> v1:172.21.15.164:6789/0 conn(0x7f182015ef90 legacy=0x7f182006dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.134+0000 7f181f400640 1 -- 172.21.15.43:0/2811616387 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182015f370 con 0x7f182006e430 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.135+0000 7f181f400640 1 --2- 172.21.15.43:0/2811616387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f18200d34e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f182006ea00 tx=0x7f181002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b928147d8814efe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.135+0000 7f181ea00640 1 -- 172.21.15.43:0/2811616387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810036070 con 0x7f182006e430 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.135+0000 7f181ea00640 1 -- 172.21.15.43:0/2811616387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1810031600 con 0x7f182006e430 2024-04-26T20:27:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.135+0000 7f181ea00640 1 -- 172.21.15.43:0/2811616387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100319d0 con 0x7f182006e430 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 -- 172.21.15.43:0/2811616387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 msgr2=0x7f18200d34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 --2- 172.21.15.43:0/2811616387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f18200d34e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f182006ea00 tx=0x7f181002a660 comp rx=0 tx=0).stop 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 -- 172.21.15.43:0/2811616387 shutdown_connections 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 --2- 172.21.15.43:0/2811616387 >> v2:172.21.15.164:3300/0 conn(0x7f182006d520 0x7f1820070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 --2- 172.21.15.43:0/2811616387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f18200d34e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 -- 172.21.15.43:0/2811616387 >> 172.21.15.43:0/2811616387 conn(0x7f18200d29c0 msgr2=0x7f18200d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 -- 172.21.15.43:0/2811616387 shutdown_connections 2024-04-26T20:27:54.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.136+0000 7f1825400640 1 -- 172.21.15.43:0/2811616387 wait complete. 2024-04-26T20:27:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.138+0000 7f1825400640 1 Processor -- start 2024-04-26T20:27:54.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.138+0000 7f1825400640 1 -- start start 2024-04-26T20:27:54.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182006d520 0x7f1820161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182015ef90 0x7f182014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18200d2010 con 0x7f182006e430 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18200d1d30 con 0x7f182006d520 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.139+0000 7f1825400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1820070ac0 con 0x7f182015ef90 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47372/0 (socket says 172.21.15.43:47372) 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 -- 172.21.15.43:0/2901860995 learned_addr learned my addr 172.21.15.43:0/2901860995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 -- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182015ef90 msgr2=0x7f182014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f1824a00640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182015ef90 0x7f182014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181fe00640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182006d520 0x7f1820161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182015ef90 0x7f182014ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 -- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182006d520 msgr2=0x7f1820161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182006d520 0x7f1820161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.140+0000 7f181f400640 1 -- 172.21.15.43:0/2901860995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18200c7f70 con 0x7f182006e430 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f181f400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1810009f40 tx=0x7f1810004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181003f020 con 0x7f182006e430 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1810037040 con 0x7f182006e430 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810036070 con 0x7f182006e430 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18200c8250 con 0x7f182006e430 2024-04-26T20:27:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.141+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18200c8810 con 0x7f182006e430 2024-04-26T20:27:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.142+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1820069a60 con 0x7f182006e430 2024-04-26T20:27:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.143+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1810042020 con 0x7f182006e430 2024-04-26T20:27:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.144+0000 7f1817e00640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.144+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f18100ac380 con 0x7f182006e430 2024-04-26T20:27:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.145+0000 7f181fe00640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.146+0000 7f181fe00640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 0x7f17fc06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1808006fd0 tx=0x7f1808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.148+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1810049030 con 0x7f182006e430 2024-04-26T20:27:54.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.349+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18200c7d20 con 0x7f17fc06c8f0 2024-04-26T20:27:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.370+0000 7f1817e00640 1 -- 172.21.15.43:0/2901860995 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250752 (secure 0 0 0) 0x7f18200c7d20 con 0x7f17fc06c8f0 2024-04-26T20:27:54.373 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:27:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 msgr2=0x7f17fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 0x7f17fc06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1808006fd0 tx=0x7f1808008040 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 msgr2=0x7f182015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1810009f40 tx=0x7f1810004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 shutdown_connections 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f17fc06c8f0 0x7f17fc06edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182015ef90 0x7f182014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f182006e430 0x7f182015f580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 --2- 172.21.15.43:0/2901860995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182006d520 0x7f1820161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 >> 172.21.15.43:0/2901860995 conn(0x7f18200d29c0 msgr2=0x7f182015ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 shutdown_connections 2024-04-26T20:27:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.377+0000 7f1825400640 1 -- 172.21.15.43:0/2901860995 wait complete. 2024-04-26T20:27:54.399 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-26T20:27:52.898184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591838,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237216,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737504,"statfs":{"total":773094113280,"available":772851204096,"internally_reserved":0,"allocated":4509696,"data_stored":2598871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.047330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929111+0000","last_change":"2024-04-26T20:27:49.918830+0000","last_active":"2024-04-26T20:27:49.929111+0000","last_peered":"2024-04-26T20:27:49.929111+0000","last_clean":"2024-04-26T20:27:49.929111+0000","last_became_active":"2024-04-26T20:27:49.918530+0000","last_became_peered":"2024-04-26T20:27:49.918530+0000","last_unstale":"2024-04-26T20:27:49.929111+0000","last_undegraded":"2024-04-26T20:27:49.929111+0000","last_fullsized":"2024-04-26T20:27:49.929111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913015+0000","last_change":"2024-04-26T20:27:49.911220+0000","last_active":"2024-04-26T20:27:49.913015+0000","last_peered":"2024-04-26T20:27:49.913015+0000","last_clean":"2024-04-26T20:27:49.913015+0000","last_became_active":"2024-04-26T20:27:49.910949+0000","last_became_peered":"2024-04-26T20:27:49.910949+0000","last_unstale":"2024-04-26T20:27:49.913015+0000","last_undegraded":"2024-04-26T20:27:49.913015+0000","last_fullsized":"2024-04-26T20:27:49.913015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926512+0000","last_change":"2024-04-26T20:27:49.926512+0000","last_active":"2024-04-26T20:27:49.926512+0000","last_peered":"2024-04-26T20:27:49.926512+0000","last_clean":"2024-04-26T20:27:49.926512+0000","last_became_active":"2024-04-26T20:27:49.925884+0000","last_became_peered":"2024-04-26T20:27:49.925884+0000","last_unstale":"2024-04-26T20:27:49.926512+0000","last_undegraded":"2024-04-26T20:27:49.926512+0000","last_fullsized":"2024-04-26T20:27:49.926512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926623+0000","last_change":"2024-04-26T20:27:49.923178+0000","last_active":"2024-04-26T20:27:49.926623+0000","last_peered":"2024-04-26T20:27:49.926623+0000","last_clean":"2024-04-26T20:27:49.926623+0000","last_became_active":"2024-04-26T20:27:49.920812+0000","last_became_peered":"2024-04-26T20:27:49.920812+0000","last_unstale":"2024-04-26T20:27:49.926623+0000","last_undegraded":"2024-04-26T20:27:49.926623+0000","last_fullsized":"2024-04-26T20:27:49.926623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:40.526461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926391+0000","last_change":"2024-04-26T20:27:49.922091+0000","last_active":"2024-04-26T20:27:49.926391+0000","last_peered":"2024-04-26T20:27:49.926391+0000","last_clean":"2024-04-26T20:27:49.926391+0000","last_became_active":"2024-04-26T20:27:49.921810+0000","last_became_peered":"2024-04-26T20:27:49.921810+0000","last_unstale":"2024-04-26T20:27:49.926391+0000","last_undegraded":"2024-04-26T20:27:49.926391+0000","last_fullsized":"2024-04-26T20:27:49.926391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:13.735142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908694+0000","last_change":"2024-04-26T20:27:49.902670+0000","last_active":"2024-04-26T20:27:49.908694+0000","last_peered":"2024-04-26T20:27:49.908694+0000","last_clean":"2024-04-26T20:27:49.908694+0000","last_became_active":"2024-04-26T20:27:49.902336+0000","last_became_peered":"2024-04-26T20:27:49.902336+0000","last_unstale":"2024-04-26T20:27:49.908694+0000","last_undegraded":"2024-04-26T20:27:49.908694+0000","last_fullsized":"2024-04-26T20:27:49.908694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922231+0000","last_change":"2024-04-26T20:27:49.922231+0000","last_active":"2024-04-26T20:27:49.922231+0000","last_peered":"2024-04-26T20:27:49.922231+0000","last_clean":"2024-04-26T20:27:49.922231+0000","last_became_active":"2024-04-26T20:27:49.921927+0000","last_became_peered":"2024-04-26T20:27:49.921927+0000","last_unstale":"2024-04-26T20:27:49.922231+0000","last_undegraded":"2024-04-26T20:27:49.922231+0000","last_fullsized":"2024-04-26T20:27:49.922231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931179+0000","last_change":"2024-04-26T20:27:49.931179+0000","last_active":"2024-04-26T20:27:49.931179+0000","last_peered":"2024-04-26T20:27:49.931179+0000","last_clean":"2024-04-26T20:27:49.931179+0000","last_became_active":"2024-04-26T20:27:49.930033+0000","last_became_peered":"2024-04-26T20:27:49.930033+0000","last_unstale":"2024-04-26T20:27:49.931179+0000","last_undegraded":"2024-04-26T20:27:49.931179+0000","last_fullsized":"2024-04-26T20:27:49.931179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926173+0000","last_change":"2024-04-26T20:27:49.926173+0000","last_active":"2024-04-26T20:27:49.926173+0000","last_peered":"2024-04-26T20:27:49.926173+0000","last_clean":"2024-04-26T20:27:49.926173+0000","last_became_active":"2024-04-26T20:27:49.925901+0000","last_became_peered":"2024-04-26T20:27:49.925901+0000","last_unstale":"2024-04-26T20:27:49.926173+0000","last_undegraded":"2024-04-26T20:27:49.926173+0000","last_fullsized":"2024-04-26T20:27:49.926173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932243+0000","last_change":"2024-04-26T20:27:49.932243+0000","last_active":"2024-04-26T20:27:49.932243+0000","last_peered":"2024-04-26T20:27:49.932243+0000","last_clean":"2024-04-26T20:27:49.932243+0000","last_became_active":"2024-04-26T20:27:49.931924+0000","last_became_peered":"2024-04-26T20:27:49.931924+0000","last_unstale":"2024-04-26T20:27:49.932243+0000","last_undegraded":"2024-04-26T20:27:49.932243+0000","last_fullsized":"2024-04-26T20:27:49.932243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928800+0000","last_change":"2024-04-26T20:27:49.924453+0000","last_active":"2024-04-26T20:27:49.928800+0000","last_peered":"2024-04-26T20:27:49.928800+0000","last_clean":"2024-04-26T20:27:49.928800+0000","last_became_active":"2024-04-26T20:27:49.924193+0000","last_became_peered":"2024-04-26T20:27:49.924193+0000","last_unstale":"2024-04-26T20:27:49.928800+0000","last_undegraded":"2024-04-26T20:27:49.928800+0000","last_fullsized":"2024-04-26T20:27:49.928800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922276+0000","last_change":"2024-04-26T20:27:49.917930+0000","last_active":"2024-04-26T20:27:49.922276+0000","last_peered":"2024-04-26T20:27:49.922276+0000","last_clean":"2024-04-26T20:27:49.922276+0000","last_became_active":"2024-04-26T20:27:49.914720+0000","last_became_peered":"2024-04-26T20:27:49.914720+0000","last_unstale":"2024-04-26T20:27:49.922276+0000","last_undegraded":"2024-04-26T20:27:49.922276+0000","last_fullsized":"2024-04-26T20:27:49.922276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932579+0000","last_change":"2024-04-26T20:27:49.932579+0000","last_active":"2024-04-26T20:27:49.932579+0000","last_peered":"2024-04-26T20:27:49.932579+0000","last_clean":"2024-04-26T20:27:49.932579+0000","last_became_active":"2024-04-26T20:27:49.931863+0000","last_became_peered":"2024-04-26T20:27:49.931863+0000","last_unstale":"2024-04-26T20:27:49.932579+0000","last_undegraded":"2024-04-26T20:27:49.932579+0000","last_fullsized":"2024-04-26T20:27:49.932579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.891489+0000","last_change":"2024-04-26T20:27:49.891489+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.891489+0000","last_undegraded":"2024-04-26T20:27:49.891489+0000","last_fullsized":"2024-04-26T20:27:49.891489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929160+0000","last_change":"2024-04-26T20:27:49.929160+0000","last_active":"2024-04-26T20:27:49.929160+0000","last_peered":"2024-04-26T20:27:49.929160+0000","last_clean":"2024-04-26T20:27:49.929160+0000","last_became_active":"2024-04-26T20:27:49.928881+0000","last_became_peered":"2024-04-26T20:27:49.928881+0000","last_unstale":"2024-04-26T20:27:49.929160+0000","last_undegraded":"2024-04-26T20:27:49.929160+0000","last_fullsized":"2024-04-26T20:27:49.929160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913270+0000","last_change":"2024-04-26T20:27:49.913270+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913270+0000","last_undegraded":"2024-04-26T20:27:49.913270+0000","last_fullsized":"2024-04-26T20:27:49.913270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.921031+0000","last_change":"2024-04-26T20:27:49.921031+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.921031+0000","last_undegraded":"2024-04-26T20:27:49.921031+0000","last_fullsized":"2024-04-26T20:27:49.921031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927012+0000","last_change":"2024-04-26T20:27:49.927012+0000","last_active":"2024-04-26T20:27:49.927012+0000","last_peered":"2024-04-26T20:27:49.927012+0000","last_clean":"2024-04-26T20:27:49.927012+0000","last_became_active":"2024-04-26T20:27:49.926696+0000","last_became_peered":"2024-04-26T20:27:49.926696+0000","last_unstale":"2024-04-26T20:27:49.927012+0000","last_undegraded":"2024-04-26T20:27:49.927012+0000","last_fullsized":"2024-04-26T20:27:49.927012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911696+0000","last_change":"2024-04-26T20:27:49.902695+0000","last_active":"2024-04-26T20:27:49.911696+0000","last_peered":"2024-04-26T20:27:49.911696+0000","last_clean":"2024-04-26T20:27:49.911696+0000","last_became_active":"2024-04-26T20:27:49.901838+0000","last_became_peered":"2024-04-26T20:27:49.901838+0000","last_unstale":"2024-04-26T20:27:49.911696+0000","last_undegraded":"2024-04-26T20:27:49.911696+0000","last_fullsized":"2024-04-26T20:27:49.911696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:44.890223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926250+0000","last_change":"2024-04-26T20:27:49.926250+0000","last_active":"2024-04-26T20:27:49.926250+0000","last_peered":"2024-04-26T20:27:49.926250+0000","last_clean":"2024-04-26T20:27:49.926250+0000","last_became_active":"2024-04-26T20:27:49.925945+0000","last_became_peered":"2024-04-26T20:27:49.925945+0000","last_unstale":"2024-04-26T20:27:49.926250+0000","last_undegraded":"2024-04-26T20:27:49.926250+0000","last_fullsized":"2024-04-26T20:27:49.926250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928074+0000","last_change":"2024-04-26T20:27:49.928074+0000","last_active":"2024-04-26T20:27:49.928074+0000","last_peered":"2024-04-26T20:27:49.928074+0000","last_clean":"2024-04-26T20:27:49.928074+0000","last_became_active":"2024-04-26T20:27:49.920120+0000","last_became_peered":"2024-04-26T20:27:49.920120+0000","last_unstale":"2024-04-26T20:27:49.928074+0000","last_undegraded":"2024-04-26T20:27:49.928074+0000","last_fullsized":"2024-04-26T20:27:49.928074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916486+0000","last_change":"2024-04-26T20:27:49.906288+0000","last_active":"2024-04-26T20:27:49.916486+0000","last_peered":"2024-04-26T20:27:49.916486+0000","last_clean":"2024-04-26T20:27:49.916486+0000","last_became_active":"2024-04-26T20:27:49.905307+0000","last_became_peered":"2024-04-26T20:27:49.905307+0000","last_unstale":"2024-04-26T20:27:49.916486+0000","last_undegraded":"2024-04-26T20:27:49.916486+0000","last_fullsized":"2024-04-26T20:27:49.916486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913198+0000","last_change":"2024-04-26T20:27:49.913198+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913198+0000","last_undegraded":"2024-04-26T20:27:49.913198+0000","last_fullsized":"2024-04-26T20:27:49.913198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925927+0000","last_change":"2024-04-26T20:27:49.921300+0000","last_active":"2024-04-26T20:27:49.925927+0000","last_peered":"2024-04-26T20:27:49.925927+0000","last_clean":"2024-04-26T20:27:49.925927+0000","last_became_active":"2024-04-26T20:27:49.921025+0000","last_became_peered":"2024-04-26T20:27:49.921025+0000","last_unstale":"2024-04-26T20:27:49.925927+0000","last_undegraded":"2024-04-26T20:27:49.925927+0000","last_fullsized":"2024-04-26T20:27:49.925927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:04.556567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917029+0000","last_change":"2024-04-26T20:27:49.917029+0000","last_active":"2024-04-26T20:27:49.917029+0000","last_peered":"2024-04-26T20:27:49.917029+0000","last_clean":"2024-04-26T20:27:49.917029+0000","last_became_active":"2024-04-26T20:27:49.916684+0000","last_became_peered":"2024-04-26T20:27:49.916684+0000","last_unstale":"2024-04-26T20:27:49.917029+0000","last_undegraded":"2024-04-26T20:27:49.917029+0000","last_fullsized":"2024-04-26T20:27:49.917029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.893547+0000","last_change":"2024-04-26T20:27:49.893547+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.893547+0000","last_undegraded":"2024-04-26T20:27:49.893547+0000","last_fullsized":"2024-04-26T20:27:49.893547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.915570+0000","last_change":"2024-04-26T20:27:49.915570+0000","last_active":"2024-04-26T20:27:49.915570+0000","last_peered":"2024-04-26T20:27:49.915570+0000","last_clean":"2024-04-26T20:27:49.915570+0000","last_became_active":"2024-04-26T20:27:49.915277+0000","last_became_peered":"2024-04-26T20:27:49.915277+0000","last_unstale":"2024-04-26T20:27:49.915570+0000","last_undegraded":"2024-04-26T20:27:49.915570+0000","last_fullsized":"2024-04-26T20:27:49.915570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929083+0000","last_change":"2024-04-26T20:27:49.923260+0000","last_active":"2024-04-26T20:27:49.929083+0000","last_peered":"2024-04-26T20:27:49.929083+0000","last_clean":"2024-04-26T20:27:49.929083+0000","last_became_active":"2024-04-26T20:27:49.922900+0000","last_became_peered":"2024-04-26T20:27:49.922900+0000","last_unstale":"2024-04-26T20:27:49.929083+0000","last_undegraded":"2024-04-26T20:27:49.929083+0000","last_fullsized":"2024-04-26T20:27:49.929083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918298+0000","last_change":"2024-04-26T20:27:49.918298+0000","last_active":"2024-04-26T20:27:49.918298+0000","last_peered":"2024-04-26T20:27:49.918298+0000","last_clean":"2024-04-26T20:27:49.918298+0000","last_became_active":"2024-04-26T20:27:49.912487+0000","last_became_peered":"2024-04-26T20:27:49.912487+0000","last_unstale":"2024-04-26T20:27:49.918298+0000","last_undegraded":"2024-04-26T20:27:49.918298+0000","last_fullsized":"2024-04-26T20:27:49.918298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925689+0000","last_change":"2024-04-26T20:27:49.914198+0000","last_active":"2024-04-26T20:27:49.925689+0000","last_peered":"2024-04-26T20:27:49.925689+0000","last_clean":"2024-04-26T20:27:49.925689+0000","last_became_active":"2024-04-26T20:27:49.913905+0000","last_became_peered":"2024-04-26T20:27:49.913905+0000","last_unstale":"2024-04-26T20:27:49.925689+0000","last_undegraded":"2024-04-26T20:27:49.925689+0000","last_fullsized":"2024-04-26T20:27:49.925689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:34.250292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913586+0000","last_change":"2024-04-26T20:27:49.913586+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913586+0000","last_undegraded":"2024-04-26T20:27:49.913586+0000","last_fullsized":"2024-04-26T20:27:49.913586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908366+0000","last_change":"2024-04-26T20:27:49.899934+0000","last_active":"2024-04-26T20:27:49.908366+0000","last_peered":"2024-04-26T20:27:49.908366+0000","last_clean":"2024-04-26T20:27:49.908366+0000","last_became_active":"2024-04-26T20:27:49.899653+0000","last_became_peered":"2024-04-26T20:27:49.899653+0000","last_unstale":"2024-04-26T20:27:49.908366+0000","last_undegraded":"2024-04-26T20:27:49.908366+0000","last_fullsized":"2024-04-26T20:27:49.908366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.919146+0000","last_change":"2024-04-26T20:27:49.919146+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.919146+0000","last_undegraded":"2024-04-26T20:27:49.919146+0000","last_fullsized":"2024-04-26T20:27:49.919146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911517+0000","last_change":"2024-04-26T20:27:48.897659+0000","last_active":"2024-04-26T20:27:49.911517+0000","last_peered":"2024-04-26T20:27:49.911517+0000","last_clean":"2024-04-26T20:27:49.911517+0000","last_became_active":"2024-04-26T20:27:48.896450+0000","last_became_peered":"2024-04-26T20:27:48.896450+0000","last_unstale":"2024-04-26T20:27:49.911517+0000","last_undegraded":"2024-04-26T20:27:49.911517+0000","last_fullsized":"2024-04-26T20:27:49.911517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892621+0000","last_change":"2024-04-26T20:27:49.892621+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892621+0000","last_undegraded":"2024-04-26T20:27:49.892621+0000","last_fullsized":"2024-04-26T20:27:49.892621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901581+0000","last_change":"2024-04-26T20:27:48.895441+0000","last_active":"2024-04-26T20:27:49.901581+0000","last_peered":"2024-04-26T20:27:49.901581+0000","last_clean":"2024-04-26T20:27:49.901581+0000","last_became_active":"2024-04-26T20:27:48.895118+0000","last_became_peered":"2024-04-26T20:27:48.895118+0000","last_unstale":"2024-04-26T20:27:49.901581+0000","last_undegraded":"2024-04-26T20:27:49.901581+0000","last_fullsized":"2024-04-26T20:27:49.901581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.890220+0000","last_change":"2024-04-26T20:27:49.890220+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.890220+0000","last_undegraded":"2024-04-26T20:27:49.890220+0000","last_fullsized":"2024-04-26T20:27:49.890220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895459+0000","last_change":"2024-04-26T20:27:48.894724+0000","last_active":"2024-04-26T20:27:49.895459+0000","last_peered":"2024-04-26T20:27:49.895459+0000","last_clean":"2024-04-26T20:27:49.895459+0000","last_became_active":"2024-04-26T20:27:48.893381+0000","last_became_peered":"2024-04-26T20:27:48.893381+0000","last_unstale":"2024-04-26T20:27:49.895459+0000","last_undegraded":"2024-04-26T20:27:49.895459+0000","last_fullsized":"2024-04-26T20:27:49.895459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931161+0000","last_change":"2024-04-26T20:27:49.929224+0000","last_active":"2024-04-26T20:27:49.931161+0000","last_peered":"2024-04-26T20:27:49.931161+0000","last_clean":"2024-04-26T20:27:49.931161+0000","last_became_active":"2024-04-26T20:27:49.928934+0000","last_became_peered":"2024-04-26T20:27:49.928934+0000","last_unstale":"2024-04-26T20:27:49.931161+0000","last_undegraded":"2024-04-26T20:27:49.931161+0000","last_fullsized":"2024-04-26T20:27:49.931161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895394+0000","last_change":"2024-04-26T20:27:48.894846+0000","last_active":"2024-04-26T20:27:49.895394+0000","last_peered":"2024-04-26T20:27:49.895394+0000","last_clean":"2024-04-26T20:27:49.895394+0000","last_became_active":"2024-04-26T20:27:48.893698+0000","last_became_peered":"2024-04-26T20:27:48.893698+0000","last_unstale":"2024-04-26T20:27:49.895394+0000","last_undegraded":"2024-04-26T20:27:49.895394+0000","last_fullsized":"2024-04-26T20:27:49.895394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925175+0000","last_change":"2024-04-26T20:27:49.920847+0000","last_active":"2024-04-26T20:27:49.925175+0000","last_peered":"2024-04-26T20:27:49.925175+0000","last_clean":"2024-04-26T20:27:49.925175+0000","last_became_active":"2024-04-26T20:27:49.920522+0000","last_became_peered":"2024-04-26T20:27:49.920522+0000","last_unstale":"2024-04-26T20:27:49.925175+0000","last_undegraded":"2024-04-26T20:27:49.925175+0000","last_fullsized":"2024-04-26T20:27:49.925175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.930867+0000","last_change":"2024-04-26T20:27:48.891209+0000","last_active":"2024-04-26T20:27:49.930867+0000","last_peered":"2024-04-26T20:27:49.930867+0000","last_clean":"2024-04-26T20:27:49.930867+0000","last_became_active":"2024-04-26T20:27:48.890894+0000","last_became_peered":"2024-04-26T20:27:48.890894+0000","last_unstale":"2024-04-26T20:27:49.930867+0000","last_undegraded":"2024-04-26T20:27:49.930867+0000","last_fullsized":"2024-04-26T20:27:49.930867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928684+0000","last_change":"2024-04-26T20:27:49.920288+0000","last_active":"2024-04-26T20:27:49.928684+0000","last_peered":"2024-04-26T20:27:49.928684+0000","last_clean":"2024-04-26T20:27:49.928684+0000","last_became_active":"2024-04-26T20:27:49.919780+0000","last_became_peered":"2024-04-26T20:27:49.919780+0000","last_unstale":"2024-04-26T20:27:49.928684+0000","last_undegraded":"2024-04-26T20:27:49.928684+0000","last_fullsized":"2024-04-26T20:27:49.928684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922040+0000","last_change":"2024-04-26T20:27:48.897588+0000","last_active":"2024-04-26T20:27:49.922040+0000","last_peered":"2024-04-26T20:27:49.922040+0000","last_clean":"2024-04-26T20:27:49.922040+0000","last_became_active":"2024-04-26T20:27:48.896574+0000","last_became_peered":"2024-04-26T20:27:48.896574+0000","last_unstale":"2024-04-26T20:27:49.922040+0000","last_undegraded":"2024-04-26T20:27:49.922040+0000","last_fullsized":"2024-04-26T20:27:49.922040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:22.623531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.919624+0000","last_change":"2024-04-26T20:27:49.916775+0000","last_active":"2024-04-26T20:27:49.919624+0000","last_peered":"2024-04-26T20:27:49.919624+0000","last_clean":"2024-04-26T20:27:49.919624+0000","last_became_active":"2024-04-26T20:27:49.916312+0000","last_became_peered":"2024-04-26T20:27:49.916312+0000","last_unstale":"2024-04-26T20:27:49.919624+0000","last_undegraded":"2024-04-26T20:27:49.919624+0000","last_fullsized":"2024-04-26T20:27:49.919624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895250+0000","last_change":"2024-04-26T20:27:48.898227+0000","last_active":"2024-04-26T20:27:49.895250+0000","last_peered":"2024-04-26T20:27:49.895250+0000","last_clean":"2024-04-26T20:27:49.895250+0000","last_became_active":"2024-04-26T20:27:48.897196+0000","last_became_peered":"2024-04-26T20:27:48.897196+0000","last_unstale":"2024-04-26T20:27:49.895250+0000","last_undegraded":"2024-04-26T20:27:49.895250+0000","last_fullsized":"2024-04-26T20:27:49.895250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:46.834398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.900144+0000","last_change":"2024-04-26T20:27:49.900144+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.900144+0000","last_undegraded":"2024-04-26T20:27:49.900144+0000","last_fullsized":"2024-04-26T20:27:49.900144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894039+0000","last_change":"2024-04-26T20:27:48.890830+0000","last_active":"2024-04-26T20:27:49.894039+0000","last_peered":"2024-04-26T20:27:49.894039+0000","last_clean":"2024-04-26T20:27:49.894039+0000","last_became_active":"2024-04-26T20:27:48.890544+0000","last_became_peered":"2024-04-26T20:27:48.890544+0000","last_unstale":"2024-04-26T20:27:49.894039+0000","last_undegraded":"2024-04-26T20:27:49.894039+0000","last_fullsized":"2024-04-26T20:27:49.894039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927244+0000","last_change":"2024-04-26T20:27:49.923399+0000","last_active":"2024-04-26T20:27:49.927244+0000","last_peered":"2024-04-26T20:27:49.927244+0000","last_clean":"2024-04-26T20:27:49.927244+0000","last_became_active":"2024-04-26T20:27:49.923127+0000","last_became_peered":"2024-04-26T20:27:49.923127+0000","last_unstale":"2024-04-26T20:27:49.927244+0000","last_undegraded":"2024-04-26T20:27:49.927244+0000","last_fullsized":"2024-04-26T20:27:49.927244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:24.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901840+0000","last_change":"2024-04-26T20:27:48.889003+0000","last_active":"2024-04-26T20:27:49.901840+0000","last_peered":"2024-04-26T20:27:49.901840+0000","last_clean":"2024-04-26T20:27:49.901840+0000","last_became_active":"2024-04-26T20:27:48.888708+0000","last_became_peered":"2024-04-26T20:27:48.888708+0000","last_unstale":"2024-04-26T20:27:49.901840+0000","last_undegraded":"2024-04-26T20:27:49.901840+0000","last_fullsized":"2024-04-26T20:27:49.901840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:17.802067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917978+0000","last_change":"2024-04-26T20:27:49.917978+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917978+0000","last_undegraded":"2024-04-26T20:27:49.917978+0000","last_fullsized":"2024-04-26T20:27:49.917978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922887+0000","last_change":"2024-04-26T20:27:48.897655+0000","last_active":"2024-04-26T20:27:49.922887+0000","last_peered":"2024-04-26T20:27:49.922887+0000","last_clean":"2024-04-26T20:27:49.922887+0000","last_became_active":"2024-04-26T20:27:48.897232+0000","last_became_peered":"2024-04-26T20:27:48.897232+0000","last_unstale":"2024-04-26T20:27:49.922887+0000","last_undegraded":"2024-04-26T20:27:49.922887+0000","last_fullsized":"2024-04-26T20:27:49.922887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925441+0000","last_change":"2024-04-26T20:27:49.921057+0000","last_active":"2024-04-26T20:27:49.925441+0000","last_peered":"2024-04-26T20:27:49.925441+0000","last_clean":"2024-04-26T20:27:49.925441+0000","last_became_active":"2024-04-26T20:27:49.920747+0000","last_became_peered":"2024-04-26T20:27:49.920747+0000","last_unstale":"2024-04-26T20:27:49.925441+0000","last_undegraded":"2024-04-26T20:27:49.925441+0000","last_fullsized":"2024-04-26T20:27:49.925441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:40.387985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922912+0000","last_change":"2024-04-26T20:27:48.895154+0000","last_active":"2024-04-26T20:27:49.922912+0000","last_peered":"2024-04-26T20:27:49.922912+0000","last_clean":"2024-04-26T20:27:49.922912+0000","last_became_active":"2024-04-26T20:27:48.894300+0000","last_became_peered":"2024-04-26T20:27:48.894300+0000","last_unstale":"2024-04-26T20:27:49.922912+0000","last_undegraded":"2024-04-26T20:27:49.922912+0000","last_fullsized":"2024-04-26T20:27:49.922912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.920045+0000","last_change":"2024-04-26T20:27:49.920045+0000","last_active":"2024-04-26T20:27:49.920045+0000","last_peered":"2024-04-26T20:27:49.920045+0000","last_clean":"2024-04-26T20:27:49.920045+0000","last_became_active":"2024-04-26T20:27:49.919739+0000","last_became_peered":"2024-04-26T20:27:49.919739+0000","last_unstale":"2024-04-26T20:27:49.920045+0000","last_undegraded":"2024-04-26T20:27:49.920045+0000","last_fullsized":"2024-04-26T20:27:49.920045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932303+0000","last_change":"2024-04-26T20:27:48.896675+0000","last_active":"2024-04-26T20:27:49.932303+0000","last_peered":"2024-04-26T20:27:49.932303+0000","last_clean":"2024-04-26T20:27:49.932303+0000","last_became_active":"2024-04-26T20:27:48.896362+0000","last_became_peered":"2024-04-26T20:27:48.896362+0000","last_unstale":"2024-04-26T20:27:49.932303+0000","last_undegraded":"2024-04-26T20:27:49.932303+0000","last_fullsized":"2024-04-26T20:27:49.932303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924165+0000","last_change":"2024-04-26T20:27:49.920198+0000","last_active":"2024-04-26T20:27:49.924165+0000","last_peered":"2024-04-26T20:27:49.924165+0000","last_clean":"2024-04-26T20:27:49.924165+0000","last_became_active":"2024-04-26T20:27:49.919936+0000","last_became_peered":"2024-04-26T20:27:49.919936+0000","last_unstale":"2024-04-26T20:27:49.924165+0000","last_undegraded":"2024-04-26T20:27:49.924165+0000","last_fullsized":"2024-04-26T20:27:49.924165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918703+0000","last_change":"2024-04-26T20:27:48.891091+0000","last_active":"2024-04-26T20:27:49.918703+0000","last_peered":"2024-04-26T20:27:49.918703+0000","last_clean":"2024-04-26T20:27:49.918703+0000","last_became_active":"2024-04-26T20:27:48.888052+0000","last_became_peered":"2024-04-26T20:27:48.888052+0000","last_unstale":"2024-04-26T20:27:49.918703+0000","last_undegraded":"2024-04-26T20:27:49.918703+0000","last_fullsized":"2024-04-26T20:27:49.918703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:35.082176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.923837+0000","last_change":"2024-04-26T20:27:49.917196+0000","last_active":"2024-04-26T20:27:49.923837+0000","last_peered":"2024-04-26T20:27:49.923837+0000","last_clean":"2024-04-26T20:27:49.923837+0000","last_became_active":"2024-04-26T20:27:49.916941+0000","last_became_peered":"2024-04-26T20:27:49.916941+0000","last_unstale":"2024-04-26T20:27:49.923837+0000","last_undegraded":"2024-04-26T20:27:49.923837+0000","last_fullsized":"2024-04-26T20:27:49.923837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895346+0000","last_change":"2024-04-26T20:27:48.894730+0000","last_active":"2024-04-26T20:27:49.895346+0000","last_peered":"2024-04-26T20:27:49.895346+0000","last_clean":"2024-04-26T20:27:49.895346+0000","last_became_active":"2024-04-26T20:27:48.894446+0000","last_became_peered":"2024-04-26T20:27:48.894446+0000","last_unstale":"2024-04-26T20:27:49.895346+0000","last_undegraded":"2024-04-26T20:27:49.895346+0000","last_fullsized":"2024-04-26T20:27:49.895346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913576+0000","last_change":"2024-04-26T20:27:49.913576+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913576+0000","last_undegraded":"2024-04-26T20:27:49.913576+0000","last_fullsized":"2024-04-26T20:27:49.913576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904291+0000","last_change":"2024-04-26T20:27:48.886899+0000","last_active":"2024-04-26T20:27:49.904291+0000","last_peered":"2024-04-26T20:27:49.904291+0000","last_clean":"2024-04-26T20:27:49.904291+0000","last_became_active":"2024-04-26T20:27:48.886536+0000","last_became_peered":"2024-04-26T20:27:48.886536+0000","last_unstale":"2024-04-26T20:27:49.904291+0000","last_undegraded":"2024-04-26T20:27:49.904291+0000","last_fullsized":"2024-04-26T20:27:49.904291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929644+0000","last_change":"2024-04-26T20:27:49.918902+0000","last_active":"2024-04-26T20:27:49.929644+0000","last_peered":"2024-04-26T20:27:49.929644+0000","last_clean":"2024-04-26T20:27:49.929644+0000","last_became_active":"2024-04-26T20:27:49.918340+0000","last_became_peered":"2024-04-26T20:27:49.918340+0000","last_unstale":"2024-04-26T20:27:49.929644+0000","last_undegraded":"2024-04-26T20:27:49.929644+0000","last_fullsized":"2024-04-26T20:27:49.929644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917635+0000","last_change":"2024-04-26T20:27:48.886522+0000","last_active":"2024-04-26T20:27:49.917635+0000","last_peered":"2024-04-26T20:27:49.917635+0000","last_clean":"2024-04-26T20:27:49.917635+0000","last_became_active":"2024-04-26T20:27:48.885874+0000","last_became_peered":"2024-04-26T20:27:48.885874+0000","last_unstale":"2024-04-26T20:27:49.917635+0000","last_undegraded":"2024-04-26T20:27:49.917635+0000","last_fullsized":"2024-04-26T20:27:49.917635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932240+0000","last_change":"2024-04-26T20:27:49.926821+0000","last_active":"2024-04-26T20:27:49.932240+0000","last_peered":"2024-04-26T20:27:49.932240+0000","last_clean":"2024-04-26T20:27:49.932240+0000","last_became_active":"2024-04-26T20:27:49.926564+0000","last_became_peered":"2024-04-26T20:27:49.926564+0000","last_unstale":"2024-04-26T20:27:49.932240+0000","last_undegraded":"2024-04-26T20:27:49.932240+0000","last_fullsized":"2024-04-26T20:27:49.932240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924746+0000","last_change":"2024-04-26T20:27:48.900526+0000","last_active":"2024-04-26T20:27:49.924746+0000","last_peered":"2024-04-26T20:27:49.924746+0000","last_clean":"2024-04-26T20:27:49.924746+0000","last_became_active":"2024-04-26T20:27:48.900110+0000","last_became_peered":"2024-04-26T20:27:48.900110+0000","last_unstale":"2024-04-26T20:27:49.924746+0000","last_undegraded":"2024-04-26T20:27:49.924746+0000","last_fullsized":"2024-04-26T20:27:49.924746+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:26.369631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928774+0000","last_change":"2024-04-26T20:27:49.923959+0000","last_active":"2024-04-26T20:27:49.928774+0000","last_peered":"2024-04-26T20:27:49.928774+0000","last_clean":"2024-04-26T20:27:49.928774+0000","last_became_active":"2024-04-26T20:27:49.923064+0000","last_became_peered":"2024-04-26T20:27:49.923064+0000","last_unstale":"2024-04-26T20:27:49.928774+0000","last_undegraded":"2024-04-26T20:27:49.928774+0000","last_fullsized":"2024-04-26T20:27:49.928774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895794+0000","last_change":"2024-04-26T20:27:48.899650+0000","last_active":"2024-04-26T20:27:49.895794+0000","last_peered":"2024-04-26T20:27:49.895794+0000","last_clean":"2024-04-26T20:27:49.895794+0000","last_became_active":"2024-04-26T20:27:48.899036+0000","last_became_peered":"2024-04-26T20:27:48.899036+0000","last_unstale":"2024-04-26T20:27:49.895794+0000","last_undegraded":"2024-04-26T20:27:49.895794+0000","last_fullsized":"2024-04-26T20:27:49.895794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:33.237276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917237+0000","last_change":"2024-04-26T20:27:49.901218+0000","last_active":"2024-04-26T20:27:49.917237+0000","last_peered":"2024-04-26T20:27:49.917237+0000","last_clean":"2024-04-26T20:27:49.917237+0000","last_became_active":"2024-04-26T20:27:49.900732+0000","last_became_peered":"2024-04-26T20:27:49.900732+0000","last_unstale":"2024-04-26T20:27:49.917237+0000","last_undegraded":"2024-04-26T20:27:49.917237+0000","last_fullsized":"2024-04-26T20:27:49.917237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926905+0000","last_change":"2024-04-26T20:27:48.883412+0000","last_active":"2024-04-26T20:27:49.926905+0000","last_peered":"2024-04-26T20:27:49.926905+0000","last_clean":"2024-04-26T20:27:49.926905+0000","last_became_active":"2024-04-26T20:27:48.882939+0000","last_became_peered":"2024-04-26T20:27:48.882939+0000","last_unstale":"2024-04-26T20:27:49.926905+0000","last_undegraded":"2024-04-26T20:27:49.926905+0000","last_fullsized":"2024-04-26T20:27:49.926905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.174009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911670+0000","last_change":"2024-04-26T20:27:49.909503+0000","last_active":"2024-04-26T20:27:49.911670+0000","last_peered":"2024-04-26T20:27:49.911670+0000","last_clean":"2024-04-26T20:27:49.911670+0000","last_became_active":"2024-04-26T20:27:49.909122+0000","last_became_peered":"2024-04-26T20:27:49.909122+0000","last_unstale":"2024-04-26T20:27:49.911670+0000","last_undegraded":"2024-04-26T20:27:49.911670+0000","last_fullsized":"2024-04-26T20:27:49.911670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:46.024331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895256+0000","last_change":"2024-04-26T20:27:48.885014+0000","last_active":"2024-04-26T20:27:49.895256+0000","last_peered":"2024-04-26T20:27:49.895256+0000","last_clean":"2024-04-26T20:27:49.895256+0000","last_became_active":"2024-04-26T20:27:48.884727+0000","last_became_peered":"2024-04-26T20:27:48.884727+0000","last_unstale":"2024-04-26T20:27:49.895256+0000","last_undegraded":"2024-04-26T20:27:49.895256+0000","last_fullsized":"2024-04-26T20:27:49.895256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917288+0000","last_change":"2024-04-26T20:27:49.917288+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917288+0000","last_undegraded":"2024-04-26T20:27:49.917288+0000","last_fullsized":"2024-04-26T20:27:49.917288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916870+0000","last_change":"2024-04-26T20:27:48.880032+0000","last_active":"2024-04-26T20:27:49.916870+0000","last_peered":"2024-04-26T20:27:49.916870+0000","last_clean":"2024-04-26T20:27:49.916870+0000","last_became_active":"2024-04-26T20:27:48.878893+0000","last_became_peered":"2024-04-26T20:27:48.878893+0000","last_unstale":"2024-04-26T20:27:49.916870+0000","last_undegraded":"2024-04-26T20:27:49.916870+0000","last_fullsized":"2024-04-26T20:27:49.916870+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:04.775515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927225+0000","last_change":"2024-04-26T20:27:49.927225+0000","last_active":"2024-04-26T20:27:49.927225+0000","last_peered":"2024-04-26T20:27:49.927225+0000","last_clean":"2024-04-26T20:27:49.927225+0000","last_became_active":"2024-04-26T20:27:49.925074+0000","last_became_peered":"2024-04-26T20:27:49.925074+0000","last_unstale":"2024-04-26T20:27:49.927225+0000","last_undegraded":"2024-04-26T20:27:49.927225+0000","last_fullsized":"2024-04-26T20:27:49.927225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886368+0000","last_change":"2024-04-26T20:27:48.896812+0000","last_active":"2024-04-26T20:27:49.886368+0000","last_peered":"2024-04-26T20:27:49.886368+0000","last_clean":"2024-04-26T20:27:49.886368+0000","last_became_active":"2024-04-26T20:27:48.895863+0000","last_became_peered":"2024-04-26T20:27:48.895863+0000","last_unstale":"2024-04-26T20:27:49.886368+0000","last_undegraded":"2024-04-26T20:27:49.886368+0000","last_fullsized":"2024-04-26T20:27:49.886368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:43.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927088+0000","last_change":"2024-04-26T20:27:49.927088+0000","last_active":"2024-04-26T20:27:49.927088+0000","last_peered":"2024-04-26T20:27:49.927088+0000","last_clean":"2024-04-26T20:27:49.927088+0000","last_became_active":"2024-04-26T20:27:49.926795+0000","last_became_peered":"2024-04-26T20:27:49.926795+0000","last_unstale":"2024-04-26T20:27:49.927088+0000","last_undegraded":"2024-04-26T20:27:49.927088+0000","last_fullsized":"2024-04-26T20:27:49.927088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913876+0000","last_change":"2024-04-26T20:27:48.891475+0000","last_active":"2024-04-26T20:27:49.913876+0000","last_peered":"2024-04-26T20:27:49.913876+0000","last_clean":"2024-04-26T20:27:49.913876+0000","last_became_active":"2024-04-26T20:27:48.891058+0000","last_became_peered":"2024-04-26T20:27:48.891058+0000","last_unstale":"2024-04-26T20:27:49.913876+0000","last_undegraded":"2024-04-26T20:27:49.913876+0000","last_fullsized":"2024-04-26T20:27:49.913876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:23.923671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929667+0000","last_change":"2024-04-26T20:27:49.921492+0000","last_active":"2024-04-26T20:27:49.929667+0000","last_peered":"2024-04-26T20:27:49.929667+0000","last_clean":"2024-04-26T20:27:49.929667+0000","last_became_active":"2024-04-26T20:27:49.921208+0000","last_became_peered":"2024-04-26T20:27:49.921208+0000","last_unstale":"2024-04-26T20:27:49.929667+0000","last_undegraded":"2024-04-26T20:27:49.929667+0000","last_fullsized":"2024-04-26T20:27:49.929667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886522+0000","last_change":"2024-04-26T20:27:48.879281+0000","last_active":"2024-04-26T20:27:49.886522+0000","last_peered":"2024-04-26T20:27:49.886522+0000","last_clean":"2024-04-26T20:27:49.886522+0000","last_became_active":"2024-04-26T20:27:48.878964+0000","last_became_peered":"2024-04-26T20:27:48.878964+0000","last_unstale":"2024-04-26T20:27:49.886522+0000","last_undegraded":"2024-04-26T20:27:49.886522+0000","last_fullsized":"2024-04-26T20:27:49.886522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:32.854744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892290+0000","last_change":"2024-04-26T20:27:49.892290+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892290+0000","last_undegraded":"2024-04-26T20:27:49.892290+0000","last_fullsized":"2024-04-26T20:27:49.892290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924918+0000","last_change":"2024-04-26T20:27:48.896599+0000","last_active":"2024-04-26T20:27:49.924918+0000","last_peered":"2024-04-26T20:27:49.924918+0000","last_clean":"2024-04-26T20:27:49.924918+0000","last_became_active":"2024-04-26T20:27:48.896224+0000","last_became_peered":"2024-04-26T20:27:48.896224+0000","last_unstale":"2024-04-26T20:27:49.924918+0000","last_undegraded":"2024-04-26T20:27:49.924918+0000","last_fullsized":"2024-04-26T20:27:49.924918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:52.021385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.918360+0000","last_change":"2024-04-26T20:27:49.918360+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.918360+0000","last_undegraded":"2024-04-26T20:27:49.918360+0000","last_fullsized":"2024-04-26T20:27:49.918360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.887587+0000","last_change":"2024-04-26T20:27:48.892674+0000","last_active":"2024-04-26T20:27:49.887587+0000","last_peered":"2024-04-26T20:27:49.887587+0000","last_clean":"2024-04-26T20:27:49.887587+0000","last_became_active":"2024-04-26T20:27:48.892364+0000","last_became_peered":"2024-04-26T20:27:48.892364+0000","last_unstale":"2024-04-26T20:27:49.887587+0000","last_undegraded":"2024-04-26T20:27:49.887587+0000","last_fullsized":"2024-04-26T20:27:49.887587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:56.403870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925751+0000","last_change":"2024-04-26T20:27:49.918395+0000","last_active":"2024-04-26T20:27:49.925751+0000","last_peered":"2024-04-26T20:27:49.925751+0000","last_clean":"2024-04-26T20:27:49.925751+0000","last_became_active":"2024-04-26T20:27:49.918111+0000","last_became_peered":"2024-04-26T20:27:49.918111+0000","last_unstale":"2024-04-26T20:27:49.925751+0000","last_undegraded":"2024-04-26T20:27:49.925751+0000","last_fullsized":"2024-04-26T20:27:49.925751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.903165+0000","last_change":"2024-04-26T20:27:48.886260+0000","last_active":"2024-04-26T20:27:49.903165+0000","last_peered":"2024-04-26T20:27:49.903165+0000","last_clean":"2024-04-26T20:27:49.903165+0000","last_became_active":"2024-04-26T20:27:48.885935+0000","last_became_peered":"2024-04-26T20:27:48.885935+0000","last_unstale":"2024-04-26T20:27:49.903165+0000","last_undegraded":"2024-04-26T20:27:49.903165+0000","last_fullsized":"2024-04-26T20:27:49.903165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924077+0000","last_change":"2024-04-26T20:27:49.919086+0000","last_active":"2024-04-26T20:27:49.924077+0000","last_peered":"2024-04-26T20:27:49.924077+0000","last_clean":"2024-04-26T20:27:49.924077+0000","last_became_active":"2024-04-26T20:27:49.913716+0000","last_became_peered":"2024-04-26T20:27:49.913716+0000","last_unstale":"2024-04-26T20:27:49.924077+0000","last_undegraded":"2024-04-26T20:27:49.924077+0000","last_fullsized":"2024-04-26T20:27:49.924077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901017+0000","last_change":"2024-04-26T20:27:48.892408+0000","last_active":"2024-04-26T20:27:49.901017+0000","last_peered":"2024-04-26T20:27:49.901017+0000","last_clean":"2024-04-26T20:27:49.901017+0000","last_became_active":"2024-04-26T20:27:48.891998+0000","last_became_peered":"2024-04-26T20:27:48.891998+0000","last_unstale":"2024-04-26T20:27:49.901017+0000","last_undegraded":"2024-04-26T20:27:49.901017+0000","last_fullsized":"2024-04-26T20:27:49.901017+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.912958+0000","last_change":"2024-04-26T20:27:49.912958+0000","last_active":"2024-04-26T20:27:49.912958+0000","last_peered":"2024-04-26T20:27:49.912958+0000","last_clean":"2024-04-26T20:27:49.912958+0000","last_became_active":"2024-04-26T20:27:49.912656+0000","last_became_peered":"2024-04-26T20:27:49.912656+0000","last_unstale":"2024-04-26T20:27:49.912958+0000","last_undegraded":"2024-04-26T20:27:49.912958+0000","last_fullsized":"2024-04-26T20:27:49.912958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904813+0000","last_change":"2024-04-26T20:27:48.883806+0000","last_active":"2024-04-26T20:27:49.904813+0000","last_peered":"2024-04-26T20:27:49.904813+0000","last_clean":"2024-04-26T20:27:49.904813+0000","last_became_active":"2024-04-26T20:27:48.883495+0000","last_became_peered":"2024-04-26T20:27:48.883495+0000","last_unstale":"2024-04-26T20:27:49.904813+0000","last_undegraded":"2024-04-26T20:27:49.904813+0000","last_fullsized":"2024-04-26T20:27:49.904813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918725+0000","last_change":"2024-04-26T20:27:49.907230+0000","last_active":"2024-04-26T20:27:49.918725+0000","last_peered":"2024-04-26T20:27:49.918725+0000","last_clean":"2024-04-26T20:27:49.918725+0000","last_became_active":"2024-04-26T20:27:49.905877+0000","last_became_peered":"2024-04-26T20:27:49.905877+0000","last_unstale":"2024-04-26T20:27:49.918725+0000","last_undegraded":"2024-04-26T20:27:49.918725+0000","last_fullsized":"2024-04-26T20:27:49.918725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894422+0000","last_change":"2024-04-26T20:27:48.893801+0000","last_active":"2024-04-26T20:27:49.894422+0000","last_peered":"2024-04-26T20:27:49.894422+0000","last_clean":"2024-04-26T20:27:49.894422+0000","last_became_active":"2024-04-26T20:27:48.893424+0000","last_became_peered":"2024-04-26T20:27:48.893424+0000","last_unstale":"2024-04-26T20:27:49.894422+0000","last_undegraded":"2024-04-26T20:27:49.894422+0000","last_fullsized":"2024-04-26T20:27:49.894422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917924+0000","last_change":"2024-04-26T20:27:49.917924+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917924+0000","last_undegraded":"2024-04-26T20:27:49.917924+0000","last_fullsized":"2024-04-26T20:27:49.917924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.898659+0000","last_change":"2024-04-26T20:27:48.885205+0000","last_active":"2024-04-26T20:27:49.898659+0000","last_peered":"2024-04-26T20:27:49.898659+0000","last_clean":"2024-04-26T20:27:49.898659+0000","last_became_active":"2024-04-26T20:27:48.884908+0000","last_became_peered":"2024-04-26T20:27:48.884908+0000","last_unstale":"2024-04-26T20:27:49.898659+0000","last_undegraded":"2024-04-26T20:27:49.898659+0000","last_fullsized":"2024-04-26T20:27:49.898659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925449+0000","last_change":"2024-04-26T20:27:49.925449+0000","last_active":"2024-04-26T20:27:49.925449+0000","last_peered":"2024-04-26T20:27:49.925449+0000","last_clean":"2024-04-26T20:27:49.925449+0000","last_became_active":"2024-04-26T20:27:49.925098+0000","last_became_peered":"2024-04-26T20:27:49.925098+0000","last_unstale":"2024-04-26T20:27:49.925449+0000","last_undegraded":"2024-04-26T20:27:49.925449+0000","last_fullsized":"2024-04-26T20:27:49.925449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918901+0000","last_change":"2024-04-26T20:27:48.889736+0000","last_active":"2024-04-26T20:27:49.918901+0000","last_peered":"2024-04-26T20:27:49.918901+0000","last_clean":"2024-04-26T20:27:49.918901+0000","last_became_active":"2024-04-26T20:27:48.889103+0000","last_became_peered":"2024-04-26T20:27:48.889103+0000","last_unstale":"2024-04-26T20:27:49.918901+0000","last_undegraded":"2024-04-26T20:27:49.918901+0000","last_fullsized":"2024-04-26T20:27:49.918901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917420+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:27:49.917420+0000","last_peered":"2024-04-26T20:27:49.917420+0000","last_clean":"2024-04-26T20:27:49.917420+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:27:49.917420+0000","last_undegraded":"2024-04-26T20:27:49.917420+0000","last_fullsized":"2024-04-26T20:27:49.917420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":766739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:27:54.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:27:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.655+0000 7f7447000640 1 Processor -- start 2024-04-26T20:27:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.655+0000 7f7447000640 1 -- start start 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.656+0000 7f7447000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7440145330 0x7f74400b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.656+0000 7f7447000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7440145710 con 0x7f74401628e0 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.656+0000 7f7447000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7440073dc0 con 0x7f74400afe30 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.656+0000 7f7447000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f74400b0250 con 0x7f7440145330 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f7445200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f74401628e0 0x7f74400af720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37806/0 (socket says 172.21.15.43:37806) 2024-04-26T20:27:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f7445200640 1 -- 172.21.15.43:0/3158496712 learned_addr learned my addr 172.21.15.43:0/3158496712 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:27:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127965473 0 0) 0x7f7440145710 con 0x7f74401628e0 2024-04-26T20:27:54.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f7445c00640 1 --2- 172.21.15.43:0/3158496712 >> v2:172.21.15.164:3300/0 conn(0x7f7440145330 0x7f74400b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7428003520 con 0x7f74401628e0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 818503694 0 0) 0x7f7428003520 con 0x7f74401628e0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.657+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 >> v2:172.21.15.164:3300/0 conn(0x7f7440145330 msgr2=0x7f74400b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 --2- 172.21.15.43:0/3158496712 >> v2:172.21.15.164:3300/0 conn(0x7f7440145330 0x7f74400b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 >> v1:172.21.15.164:6789/0 conn(0x7f74400afe30 legacy=0x7f7440073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400ae960 con 0x7f74401628e0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f742c001670 con 0x7f74401628e0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 >> v1:172.21.15.43:6789/0 conn(0x7f74401628e0 legacy=0x7f74400af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74280039b0 0x7f7428003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 0x7f742800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74400b0250 con 0x7f74280051c0 2024-04-26T20:27:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f742800c090 con 0x7f74280039b0 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f742800c290 con 0x7f7428004800 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f7446600640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 0x7f742800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.658+0000 7f7445200640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7445c00640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74280039b0 0x7f7428003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7446600640 1 -- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7428004800 msgr2=0x7f7428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7446600640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7446600640 1 -- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74280039b0 msgr2=0x7f7428003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7446600640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74280039b0 0x7f7428003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.659+0000 7f7446600640 1 -- 172.21.15.43:0/3158496712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7428003520 con 0x7f74280051c0 2024-04-26T20:27:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.660+0000 7f7446600640 1 -- 172.21.15.43:0/3158496712 reap_dead start 2024-04-26T20:27:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.660+0000 7f7446600640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 0x7f742800b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7438001e50 tx=0x7f7428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b279d87570ff76ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.660+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438001a70 con 0x7f74280051c0 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.660+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438003970 con 0x7f74280051c0 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.660+0000 7f743fe00640 1 -- 172.21.15.43:0/3158496712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438003d60 con 0x7f74280051c0 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 -- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 msgr2=0x7f742800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 0x7f742800b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7438001e50 tx=0x7f7428005830 comp rx=0 tx=0).stop 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 -- 172.21.15.43:0/3158496712 shutdown_connections 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 --2- 172.21.15.43:0/3158496712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74280051c0 0x7f742800b9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 -- 172.21.15.43:0/3158496712 >> 172.21.15.43:0/3158496712 conn(0x7f7440068090 msgr2=0x7f7440068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 -- 172.21.15.43:0/3158496712 shutdown_connections 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.661+0000 7f7447000640 1 -- 172.21.15.43:0/3158496712 wait complete. 2024-04-26T20:27:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.663+0000 7f7447000640 1 Processor -- start 2024-04-26T20:27:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.664+0000 7f7447000640 1 -- start start 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74280051c0 0x7f744015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74400b19d0 0x7f74400b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f74400af200 con 0x7f744015cf10 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74400aef20 con 0x7f74400b19d0 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7447000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7440162d70 con 0x7f74280051c0 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7445200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7445200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47394/0 (socket says 172.21.15.43:47394) 2024-04-26T20:27:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.665+0000 7f7445200640 1 -- 172.21.15.43:0/357899306 learned_addr learned my addr 172.21.15.43:0/357899306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445c00640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74280051c0 0x7f744015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7446600640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74400b19d0 0x7f74400b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 -- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74280051c0 msgr2=0x7f744015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74280051c0 0x7f744015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 -- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74400b19d0 msgr2=0x7f74400b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74400b19d0 0x7f74400b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 -- 172.21.15.43:0/357899306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400b23c0 con 0x7f744015cf10 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7445200640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f742c000f60 tx=0x7f742c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00ee70 con 0x7f744015cf10 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c018070 con 0x7f744015cf10 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00cad0 con 0x7f744015cf10 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.666+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74400b37d0 con 0x7f744015cf10 2024-04-26T20:27:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.667+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74400b3d90 con 0x7f744015cf10 2024-04-26T20:27:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.668+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440069350 con 0x7f744015cf10 2024-04-26T20:27:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.669+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f742c00cc70 con 0x7f744015cf10 2024-04-26T20:27:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.670+0000 7f743d600640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 0x7f742006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.671+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f742c08bb20 con 0x7f744015cf10 2024-04-26T20:27:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.671+0000 7f7445c00640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 0x7f742006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.672+0000 7f7445c00640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 0x7f742006ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7440073630 tx=0x7f7434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.676+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f742c059dc0 con 0x7f744015cf10 2024-04-26T20:27:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.875+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f744015d8d0 con 0x7f742006c920 2024-04-26T20:27:54.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.883+0000 7f743d600640 1 -- 172.21.15.43:0/357899306 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250752 (secure 0 0 0) 0x7f7434004030 con 0x7f742006c920 2024-04-26T20:27:54.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:54.890 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:27:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 msgr2=0x7f742006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 0x7f742006ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7440073630 tx=0x7f7434008040 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 msgr2=0x7f744015d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f742c000f60 tx=0x7f742c00ec80 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 shutdown_connections 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f742006c920 0x7f742006ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74400b19d0 0x7f74400b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f744015cf10 0x7f744015d390 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 --2- 172.21.15.43:0/357899306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74280051c0 0x7f744015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 >> 172.21.15.43:0/357899306 conn(0x7f7440068090 msgr2=0x7f74400ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:54.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 shutdown_connections 2024-04-26T20:27:54.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:54.891+0000 7f7447000640 1 -- 172.21.15.43:0/357899306 wait complete. 2024-04-26T20:27:54.914 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-26T20:27:52.898184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591838,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237216,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737504,"statfs":{"total":773094113280,"available":772851204096,"internally_reserved":0,"allocated":4509696,"data_stored":2598871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.047330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929111+0000","last_change":"2024-04-26T20:27:49.918830+0000","last_active":"2024-04-26T20:27:49.929111+0000","last_peered":"2024-04-26T20:27:49.929111+0000","last_clean":"2024-04-26T20:27:49.929111+0000","last_became_active":"2024-04-26T20:27:49.918530+0000","last_became_peered":"2024-04-26T20:27:49.918530+0000","last_unstale":"2024-04-26T20:27:49.929111+0000","last_undegraded":"2024-04-26T20:27:49.929111+0000","last_fullsized":"2024-04-26T20:27:49.929111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913015+0000","last_change":"2024-04-26T20:27:49.911220+0000","last_active":"2024-04-26T20:27:49.913015+0000","last_peered":"2024-04-26T20:27:49.913015+0000","last_clean":"2024-04-26T20:27:49.913015+0000","last_became_active":"2024-04-26T20:27:49.910949+0000","last_became_peered":"2024-04-26T20:27:49.910949+0000","last_unstale":"2024-04-26T20:27:49.913015+0000","last_undegraded":"2024-04-26T20:27:49.913015+0000","last_fullsized":"2024-04-26T20:27:49.913015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926512+0000","last_change":"2024-04-26T20:27:49.926512+0000","last_active":"2024-04-26T20:27:49.926512+0000","last_peered":"2024-04-26T20:27:49.926512+0000","last_clean":"2024-04-26T20:27:49.926512+0000","last_became_active":"2024-04-26T20:27:49.925884+0000","last_became_peered":"2024-04-26T20:27:49.925884+0000","last_unstale":"2024-04-26T20:27:49.926512+0000","last_undegraded":"2024-04-26T20:27:49.926512+0000","last_fullsized":"2024-04-26T20:27:49.926512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926623+0000","last_change":"2024-04-26T20:27:49.923178+0000","last_active":"2024-04-26T20:27:49.926623+0000","last_peered":"2024-04-26T20:27:49.926623+0000","last_clean":"2024-04-26T20:27:49.926623+0000","last_became_active":"2024-04-26T20:27:49.920812+0000","last_became_peered":"2024-04-26T20:27:49.920812+0000","last_unstale":"2024-04-26T20:27:49.926623+0000","last_undegraded":"2024-04-26T20:27:49.926623+0000","last_fullsized":"2024-04-26T20:27:49.926623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:40.526461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926391+0000","last_change":"2024-04-26T20:27:49.922091+0000","last_active":"2024-04-26T20:27:49.926391+0000","last_peered":"2024-04-26T20:27:49.926391+0000","last_clean":"2024-04-26T20:27:49.926391+0000","last_became_active":"2024-04-26T20:27:49.921810+0000","last_became_peered":"2024-04-26T20:27:49.921810+0000","last_unstale":"2024-04-26T20:27:49.926391+0000","last_undegraded":"2024-04-26T20:27:49.926391+0000","last_fullsized":"2024-04-26T20:27:49.926391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:13.735142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908694+0000","last_change":"2024-04-26T20:27:49.902670+0000","last_active":"2024-04-26T20:27:49.908694+0000","last_peered":"2024-04-26T20:27:49.908694+0000","last_clean":"2024-04-26T20:27:49.908694+0000","last_became_active":"2024-04-26T20:27:49.902336+0000","last_became_peered":"2024-04-26T20:27:49.902336+0000","last_unstale":"2024-04-26T20:27:49.908694+0000","last_undegraded":"2024-04-26T20:27:49.908694+0000","last_fullsized":"2024-04-26T20:27:49.908694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922231+0000","last_change":"2024-04-26T20:27:49.922231+0000","last_active":"2024-04-26T20:27:49.922231+0000","last_peered":"2024-04-26T20:27:49.922231+0000","last_clean":"2024-04-26T20:27:49.922231+0000","last_became_active":"2024-04-26T20:27:49.921927+0000","last_became_peered":"2024-04-26T20:27:49.921927+0000","last_unstale":"2024-04-26T20:27:49.922231+0000","last_undegraded":"2024-04-26T20:27:49.922231+0000","last_fullsized":"2024-04-26T20:27:49.922231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931179+0000","last_change":"2024-04-26T20:27:49.931179+0000","last_active":"2024-04-26T20:27:49.931179+0000","last_peered":"2024-04-26T20:27:49.931179+0000","last_clean":"2024-04-26T20:27:49.931179+0000","last_became_active":"2024-04-26T20:27:49.930033+0000","last_became_peered":"2024-04-26T20:27:49.930033+0000","last_unstale":"2024-04-26T20:27:49.931179+0000","last_undegraded":"2024-04-26T20:27:49.931179+0000","last_fullsized":"2024-04-26T20:27:49.931179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926173+0000","last_change":"2024-04-26T20:27:49.926173+0000","last_active":"2024-04-26T20:27:49.926173+0000","last_peered":"2024-04-26T20:27:49.926173+0000","last_clean":"2024-04-26T20:27:49.926173+0000","last_became_active":"2024-04-26T20:27:49.925901+0000","last_became_peered":"2024-04-26T20:27:49.925901+0000","last_unstale":"2024-04-26T20:27:49.926173+0000","last_undegraded":"2024-04-26T20:27:49.926173+0000","last_fullsized":"2024-04-26T20:27:49.926173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932243+0000","last_change":"2024-04-26T20:27:49.932243+0000","last_active":"2024-04-26T20:27:49.932243+0000","last_peered":"2024-04-26T20:27:49.932243+0000","last_clean":"2024-04-26T20:27:49.932243+0000","last_became_active":"2024-04-26T20:27:49.931924+0000","last_became_peered":"2024-04-26T20:27:49.931924+0000","last_unstale":"2024-04-26T20:27:49.932243+0000","last_undegraded":"2024-04-26T20:27:49.932243+0000","last_fullsized":"2024-04-26T20:27:49.932243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928800+0000","last_change":"2024-04-26T20:27:49.924453+0000","last_active":"2024-04-26T20:27:49.928800+0000","last_peered":"2024-04-26T20:27:49.928800+0000","last_clean":"2024-04-26T20:27:49.928800+0000","last_became_active":"2024-04-26T20:27:49.924193+0000","last_became_peered":"2024-04-26T20:27:49.924193+0000","last_unstale":"2024-04-26T20:27:49.928800+0000","last_undegraded":"2024-04-26T20:27:49.928800+0000","last_fullsized":"2024-04-26T20:27:49.928800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922276+0000","last_change":"2024-04-26T20:27:49.917930+0000","last_active":"2024-04-26T20:27:49.922276+0000","last_peered":"2024-04-26T20:27:49.922276+0000","last_clean":"2024-04-26T20:27:49.922276+0000","last_became_active":"2024-04-26T20:27:49.914720+0000","last_became_peered":"2024-04-26T20:27:49.914720+0000","last_unstale":"2024-04-26T20:27:49.922276+0000","last_undegraded":"2024-04-26T20:27:49.922276+0000","last_fullsized":"2024-04-26T20:27:49.922276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932579+0000","last_change":"2024-04-26T20:27:49.932579+0000","last_active":"2024-04-26T20:27:49.932579+0000","last_peered":"2024-04-26T20:27:49.932579+0000","last_clean":"2024-04-26T20:27:49.932579+0000","last_became_active":"2024-04-26T20:27:49.931863+0000","last_became_peered":"2024-04-26T20:27:49.931863+0000","last_unstale":"2024-04-26T20:27:49.932579+0000","last_undegraded":"2024-04-26T20:27:49.932579+0000","last_fullsized":"2024-04-26T20:27:49.932579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.891489+0000","last_change":"2024-04-26T20:27:49.891489+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.891489+0000","last_undegraded":"2024-04-26T20:27:49.891489+0000","last_fullsized":"2024-04-26T20:27:49.891489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929160+0000","last_change":"2024-04-26T20:27:49.929160+0000","last_active":"2024-04-26T20:27:49.929160+0000","last_peered":"2024-04-26T20:27:49.929160+0000","last_clean":"2024-04-26T20:27:49.929160+0000","last_became_active":"2024-04-26T20:27:49.928881+0000","last_became_peered":"2024-04-26T20:27:49.928881+0000","last_unstale":"2024-04-26T20:27:49.929160+0000","last_undegraded":"2024-04-26T20:27:49.929160+0000","last_fullsized":"2024-04-26T20:27:49.929160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913270+0000","last_change":"2024-04-26T20:27:49.913270+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913270+0000","last_undegraded":"2024-04-26T20:27:49.913270+0000","last_fullsized":"2024-04-26T20:27:49.913270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.921031+0000","last_change":"2024-04-26T20:27:49.921031+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.921031+0000","last_undegraded":"2024-04-26T20:27:49.921031+0000","last_fullsized":"2024-04-26T20:27:49.921031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927012+0000","last_change":"2024-04-26T20:27:49.927012+0000","last_active":"2024-04-26T20:27:49.927012+0000","last_peered":"2024-04-26T20:27:49.927012+0000","last_clean":"2024-04-26T20:27:49.927012+0000","last_became_active":"2024-04-26T20:27:49.926696+0000","last_became_peered":"2024-04-26T20:27:49.926696+0000","last_unstale":"2024-04-26T20:27:49.927012+0000","last_undegraded":"2024-04-26T20:27:49.927012+0000","last_fullsized":"2024-04-26T20:27:49.927012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911696+0000","last_change":"2024-04-26T20:27:49.902695+0000","last_active":"2024-04-26T20:27:49.911696+0000","last_peered":"2024-04-26T20:27:49.911696+0000","last_clean":"2024-04-26T20:27:49.911696+0000","last_became_active":"2024-04-26T20:27:49.901838+0000","last_became_peered":"2024-04-26T20:27:49.901838+0000","last_unstale":"2024-04-26T20:27:49.911696+0000","last_undegraded":"2024-04-26T20:27:49.911696+0000","last_fullsized":"2024-04-26T20:27:49.911696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:44.890223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926250+0000","last_change":"2024-04-26T20:27:49.926250+0000","last_active":"2024-04-26T20:27:49.926250+0000","last_peered":"2024-04-26T20:27:49.926250+0000","last_clean":"2024-04-26T20:27:49.926250+0000","last_became_active":"2024-04-26T20:27:49.925945+0000","last_became_peered":"2024-04-26T20:27:49.925945+0000","last_unstale":"2024-04-26T20:27:49.926250+0000","last_undegraded":"2024-04-26T20:27:49.926250+0000","last_fullsized":"2024-04-26T20:27:49.926250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928074+0000","last_change":"2024-04-26T20:27:49.928074+0000","last_active":"2024-04-26T20:27:49.928074+0000","last_peered":"2024-04-26T20:27:49.928074+0000","last_clean":"2024-04-26T20:27:49.928074+0000","last_became_active":"2024-04-26T20:27:49.920120+0000","last_became_peered":"2024-04-26T20:27:49.920120+0000","last_unstale":"2024-04-26T20:27:49.928074+0000","last_undegraded":"2024-04-26T20:27:49.928074+0000","last_fullsized":"2024-04-26T20:27:49.928074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916486+0000","last_change":"2024-04-26T20:27:49.906288+0000","last_active":"2024-04-26T20:27:49.916486+0000","last_peered":"2024-04-26T20:27:49.916486+0000","last_clean":"2024-04-26T20:27:49.916486+0000","last_became_active":"2024-04-26T20:27:49.905307+0000","last_became_peered":"2024-04-26T20:27:49.905307+0000","last_unstale":"2024-04-26T20:27:49.916486+0000","last_undegraded":"2024-04-26T20:27:49.916486+0000","last_fullsized":"2024-04-26T20:27:49.916486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913198+0000","last_change":"2024-04-26T20:27:49.913198+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913198+0000","last_undegraded":"2024-04-26T20:27:49.913198+0000","last_fullsized":"2024-04-26T20:27:49.913198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925927+0000","last_change":"2024-04-26T20:27:49.921300+0000","last_active":"2024-04-26T20:27:49.925927+0000","last_peered":"2024-04-26T20:27:49.925927+0000","last_clean":"2024-04-26T20:27:49.925927+0000","last_became_active":"2024-04-26T20:27:49.921025+0000","last_became_peered":"2024-04-26T20:27:49.921025+0000","last_unstale":"2024-04-26T20:27:49.925927+0000","last_undegraded":"2024-04-26T20:27:49.925927+0000","last_fullsized":"2024-04-26T20:27:49.925927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:04.556567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917029+0000","last_change":"2024-04-26T20:27:49.917029+0000","last_active":"2024-04-26T20:27:49.917029+0000","last_peered":"2024-04-26T20:27:49.917029+0000","last_clean":"2024-04-26T20:27:49.917029+0000","last_became_active":"2024-04-26T20:27:49.916684+0000","last_became_peered":"2024-04-26T20:27:49.916684+0000","last_unstale":"2024-04-26T20:27:49.917029+0000","last_undegraded":"2024-04-26T20:27:49.917029+0000","last_fullsized":"2024-04-26T20:27:49.917029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.893547+0000","last_change":"2024-04-26T20:27:49.893547+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.893547+0000","last_undegraded":"2024-04-26T20:27:49.893547+0000","last_fullsized":"2024-04-26T20:27:49.893547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.915570+0000","last_change":"2024-04-26T20:27:49.915570+0000","last_active":"2024-04-26T20:27:49.915570+0000","last_peered":"2024-04-26T20:27:49.915570+0000","last_clean":"2024-04-26T20:27:49.915570+0000","last_became_active":"2024-04-26T20:27:49.915277+0000","last_became_peered":"2024-04-26T20:27:49.915277+0000","last_unstale":"2024-04-26T20:27:49.915570+0000","last_undegraded":"2024-04-26T20:27:49.915570+0000","last_fullsized":"2024-04-26T20:27:49.915570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929083+0000","last_change":"2024-04-26T20:27:49.923260+0000","last_active":"2024-04-26T20:27:49.929083+0000","last_peered":"2024-04-26T20:27:49.929083+0000","last_clean":"2024-04-26T20:27:49.929083+0000","last_became_active":"2024-04-26T20:27:49.922900+0000","last_became_peered":"2024-04-26T20:27:49.922900+0000","last_unstale":"2024-04-26T20:27:49.929083+0000","last_undegraded":"2024-04-26T20:27:49.929083+0000","last_fullsized":"2024-04-26T20:27:49.929083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918298+0000","last_change":"2024-04-26T20:27:49.918298+0000","last_active":"2024-04-26T20:27:49.918298+0000","last_peered":"2024-04-26T20:27:49.918298+0000","last_clean":"2024-04-26T20:27:49.918298+0000","last_became_active":"2024-04-26T20:27:49.912487+0000","last_became_peered":"2024-04-26T20:27:49.912487+0000","last_unstale":"2024-04-26T20:27:49.918298+0000","last_undegraded":"2024-04-26T20:27:49.918298+0000","last_fullsized":"2024-04-26T20:27:49.918298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925689+0000","last_change":"2024-04-26T20:27:49.914198+0000","last_active":"2024-04-26T20:27:49.925689+0000","last_peered":"2024-04-26T20:27:49.925689+0000","last_clean":"2024-04-26T20:27:49.925689+0000","last_became_active":"2024-04-26T20:27:49.913905+0000","last_became_peered":"2024-04-26T20:27:49.913905+0000","last_unstale":"2024-04-26T20:27:49.925689+0000","last_undegraded":"2024-04-26T20:27:49.925689+0000","last_fullsized":"2024-04-26T20:27:49.925689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:34.250292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913586+0000","last_change":"2024-04-26T20:27:49.913586+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913586+0000","last_undegraded":"2024-04-26T20:27:49.913586+0000","last_fullsized":"2024-04-26T20:27:49.913586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908366+0000","last_change":"2024-04-26T20:27:49.899934+0000","last_active":"2024-04-26T20:27:49.908366+0000","last_peered":"2024-04-26T20:27:49.908366+0000","last_clean":"2024-04-26T20:27:49.908366+0000","last_became_active":"2024-04-26T20:27:49.899653+0000","last_became_peered":"2024-04-26T20:27:49.899653+0000","last_unstale":"2024-04-26T20:27:49.908366+0000","last_undegraded":"2024-04-26T20:27:49.908366+0000","last_fullsized":"2024-04-26T20:27:49.908366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.919146+0000","last_change":"2024-04-26T20:27:49.919146+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.919146+0000","last_undegraded":"2024-04-26T20:27:49.919146+0000","last_fullsized":"2024-04-26T20:27:49.919146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911517+0000","last_change":"2024-04-26T20:27:48.897659+0000","last_active":"2024-04-26T20:27:49.911517+0000","last_peered":"2024-04-26T20:27:49.911517+0000","last_clean":"2024-04-26T20:27:49.911517+0000","last_became_active":"2024-04-26T20:27:48.896450+0000","last_became_peered":"2024-04-26T20:27:48.896450+0000","last_unstale":"2024-04-26T20:27:49.911517+0000","last_undegraded":"2024-04-26T20:27:49.911517+0000","last_fullsized":"2024-04-26T20:27:49.911517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892621+0000","last_change":"2024-04-26T20:27:49.892621+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892621+0000","last_undegraded":"2024-04-26T20:27:49.892621+0000","last_fullsized":"2024-04-26T20:27:49.892621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901581+0000","last_change":"2024-04-26T20:27:48.895441+0000","last_active":"2024-04-26T20:27:49.901581+0000","last_peered":"2024-04-26T20:27:49.901581+0000","last_clean":"2024-04-26T20:27:49.901581+0000","last_became_active":"2024-04-26T20:27:48.895118+0000","last_became_peered":"2024-04-26T20:27:48.895118+0000","last_unstale":"2024-04-26T20:27:49.901581+0000","last_undegraded":"2024-04-26T20:27:49.901581+0000","last_fullsized":"2024-04-26T20:27:49.901581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.890220+0000","last_change":"2024-04-26T20:27:49.890220+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.890220+0000","last_undegraded":"2024-04-26T20:27:49.890220+0000","last_fullsized":"2024-04-26T20:27:49.890220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895459+0000","last_change":"2024-04-26T20:27:48.894724+0000","last_active":"2024-04-26T20:27:49.895459+0000","last_peered":"2024-04-26T20:27:49.895459+0000","last_clean":"2024-04-26T20:27:49.895459+0000","last_became_active":"2024-04-26T20:27:48.893381+0000","last_became_peered":"2024-04-26T20:27:48.893381+0000","last_unstale":"2024-04-26T20:27:49.895459+0000","last_undegraded":"2024-04-26T20:27:49.895459+0000","last_fullsized":"2024-04-26T20:27:49.895459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931161+0000","last_change":"2024-04-26T20:27:49.929224+0000","last_active":"2024-04-26T20:27:49.931161+0000","last_peered":"2024-04-26T20:27:49.931161+0000","last_clean":"2024-04-26T20:27:49.931161+0000","last_became_active":"2024-04-26T20:27:49.928934+0000","last_became_peered":"2024-04-26T20:27:49.928934+0000","last_unstale":"2024-04-26T20:27:49.931161+0000","last_undegraded":"2024-04-26T20:27:49.931161+0000","last_fullsized":"2024-04-26T20:27:49.931161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895394+0000","last_change":"2024-04-26T20:27:48.894846+0000","last_active":"2024-04-26T20:27:49.895394+0000","last_peered":"2024-04-26T20:27:49.895394+0000","last_clean":"2024-04-26T20:27:49.895394+0000","last_became_active":"2024-04-26T20:27:48.893698+0000","last_became_peered":"2024-04-26T20:27:48.893698+0000","last_unstale":"2024-04-26T20:27:49.895394+0000","last_undegraded":"2024-04-26T20:27:49.895394+0000","last_fullsized":"2024-04-26T20:27:49.895394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925175+0000","last_change":"2024-04-26T20:27:49.920847+0000","last_active":"2024-04-26T20:27:49.925175+0000","last_peered":"2024-04-26T20:27:49.925175+0000","last_clean":"2024-04-26T20:27:49.925175+0000","last_became_active":"2024-04-26T20:27:49.920522+0000","last_became_peered":"2024-04-26T20:27:49.920522+0000","last_unstale":"2024-04-26T20:27:49.925175+0000","last_undegraded":"2024-04-26T20:27:49.925175+0000","last_fullsized":"2024-04-26T20:27:49.925175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.930867+0000","last_change":"2024-04-26T20:27:48.891209+0000","last_active":"2024-04-26T20:27:49.930867+0000","last_peered":"2024-04-26T20:27:49.930867+0000","last_clean":"2024-04-26T20:27:49.930867+0000","last_became_active":"2024-04-26T20:27:48.890894+0000","last_became_peered":"2024-04-26T20:27:48.890894+0000","last_unstale":"2024-04-26T20:27:49.930867+0000","last_undegraded":"2024-04-26T20:27:49.930867+0000","last_fullsized":"2024-04-26T20:27:49.930867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928684+0000","last_change":"2024-04-26T20:27:49.920288+0000","last_active":"2024-04-26T20:27:49.928684+0000","last_peered":"2024-04-26T20:27:49.928684+0000","last_clean":"2024-04-26T20:27:49.928684+0000","last_became_active":"2024-04-26T20:27:49.919780+0000","last_became_peered":"2024-04-26T20:27:49.919780+0000","last_unstale":"2024-04-26T20:27:49.928684+0000","last_undegraded":"2024-04-26T20:27:49.928684+0000","last_fullsized":"2024-04-26T20:27:49.928684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922040+0000","last_change":"2024-04-26T20:27:48.897588+0000","last_active":"2024-04-26T20:27:49.922040+0000","last_peered":"2024-04-26T20:27:49.922040+0000","last_clean":"2024-04-26T20:27:49.922040+0000","last_became_active":"2024-04-26T20:27:48.896574+0000","last_became_peered":"2024-04-26T20:27:48.896574+0000","last_unstale":"2024-04-26T20:27:49.922040+0000","last_undegraded":"2024-04-26T20:27:49.922040+0000","last_fullsized":"2024-04-26T20:27:49.922040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:22.623531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.919624+0000","last_change":"2024-04-26T20:27:49.916775+0000","last_active":"2024-04-26T20:27:49.919624+0000","last_peered":"2024-04-26T20:27:49.919624+0000","last_clean":"2024-04-26T20:27:49.919624+0000","last_became_active":"2024-04-26T20:27:49.916312+0000","last_became_peered":"2024-04-26T20:27:49.916312+0000","last_unstale":"2024-04-26T20:27:49.919624+0000","last_undegraded":"2024-04-26T20:27:49.919624+0000","last_fullsized":"2024-04-26T20:27:49.919624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895250+0000","last_change":"2024-04-26T20:27:48.898227+0000","last_active":"2024-04-26T20:27:49.895250+0000","last_peered":"2024-04-26T20:27:49.895250+0000","last_clean":"2024-04-26T20:27:49.895250+0000","last_became_active":"2024-04-26T20:27:48.897196+0000","last_became_peered":"2024-04-26T20:27:48.897196+0000","last_unstale":"2024-04-26T20:27:49.895250+0000","last_undegraded":"2024-04-26T20:27:49.895250+0000","last_fullsized":"2024-04-26T20:27:49.895250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:46.834398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.900144+0000","last_change":"2024-04-26T20:27:49.900144+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.900144+0000","last_undegraded":"2024-04-26T20:27:49.900144+0000","last_fullsized":"2024-04-26T20:27:49.900144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894039+0000","last_change":"2024-04-26T20:27:48.890830+0000","last_active":"2024-04-26T20:27:49.894039+0000","last_peered":"2024-04-26T20:27:49.894039+0000","last_clean":"2024-04-26T20:27:49.894039+0000","last_became_active":"2024-04-26T20:27:48.890544+0000","last_became_peered":"2024-04-26T20:27:48.890544+0000","last_unstale":"2024-04-26T20:27:49.894039+0000","last_undegraded":"2024-04-26T20:27:49.894039+0000","last_fullsized":"2024-04-26T20:27:49.894039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927244+0000","last_change":"2024-04-26T20:27:49.923399+0000","last_active":"2024-04-26T20:27:49.927244+0000","last_peered":"2024-04-26T20:27:49.927244+0000","last_clean":"2024-04-26T20:27:49.927244+0000","last_became_active":"2024-04-26T20:27:49.923127+0000","last_became_peered":"2024-04-26T20:27:49.923127+0000","last_unstale":"2024-04-26T20:27:49.927244+0000","last_undegraded":"2024-04-26T20:27:49.927244+0000","last_fullsized":"2024-04-26T20:27:49.927244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:24.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901840+0000","last_change":"2024-04-26T20:27:48.889003+0000","last_active":"2024-04-26T20:27:49.901840+0000","last_peered":"2024-04-26T20:27:49.901840+0000","last_clean":"2024-04-26T20:27:49.901840+0000","last_became_active":"2024-04-26T20:27:48.888708+0000","last_became_peered":"2024-04-26T20:27:48.888708+0000","last_unstale":"2024-04-26T20:27:49.901840+0000","last_undegraded":"2024-04-26T20:27:49.901840+0000","last_fullsized":"2024-04-26T20:27:49.901840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:17.802067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917978+0000","last_change":"2024-04-26T20:27:49.917978+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917978+0000","last_undegraded":"2024-04-26T20:27:49.917978+0000","last_fullsized":"2024-04-26T20:27:49.917978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922887+0000","last_change":"2024-04-26T20:27:48.897655+0000","last_active":"2024-04-26T20:27:49.922887+0000","last_peered":"2024-04-26T20:27:49.922887+0000","last_clean":"2024-04-26T20:27:49.922887+0000","last_became_active":"2024-04-26T20:27:48.897232+0000","last_became_peered":"2024-04-26T20:27:48.897232+0000","last_unstale":"2024-04-26T20:27:49.922887+0000","last_undegraded":"2024-04-26T20:27:49.922887+0000","last_fullsized":"2024-04-26T20:27:49.922887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925441+0000","last_change":"2024-04-26T20:27:49.921057+0000","last_active":"2024-04-26T20:27:49.925441+0000","last_peered":"2024-04-26T20:27:49.925441+0000","last_clean":"2024-04-26T20:27:49.925441+0000","last_became_active":"2024-04-26T20:27:49.920747+0000","last_became_peered":"2024-04-26T20:27:49.920747+0000","last_unstale":"2024-04-26T20:27:49.925441+0000","last_undegraded":"2024-04-26T20:27:49.925441+0000","last_fullsized":"2024-04-26T20:27:49.925441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:40.387985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922912+0000","last_change":"2024-04-26T20:27:48.895154+0000","last_active":"2024-04-26T20:27:49.922912+0000","last_peered":"2024-04-26T20:27:49.922912+0000","last_clean":"2024-04-26T20:27:49.922912+0000","last_became_active":"2024-04-26T20:27:48.894300+0000","last_became_peered":"2024-04-26T20:27:48.894300+0000","last_unstale":"2024-04-26T20:27:49.922912+0000","last_undegraded":"2024-04-26T20:27:49.922912+0000","last_fullsized":"2024-04-26T20:27:49.922912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.920045+0000","last_change":"2024-04-26T20:27:49.920045+0000","last_active":"2024-04-26T20:27:49.920045+0000","last_peered":"2024-04-26T20:27:49.920045+0000","last_clean":"2024-04-26T20:27:49.920045+0000","last_became_active":"2024-04-26T20:27:49.919739+0000","last_became_peered":"2024-04-26T20:27:49.919739+0000","last_unstale":"2024-04-26T20:27:49.920045+0000","last_undegraded":"2024-04-26T20:27:49.920045+0000","last_fullsized":"2024-04-26T20:27:49.920045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932303+0000","last_change":"2024-04-26T20:27:48.896675+0000","last_active":"2024-04-26T20:27:49.932303+0000","last_peered":"2024-04-26T20:27:49.932303+0000","last_clean":"2024-04-26T20:27:49.932303+0000","last_became_active":"2024-04-26T20:27:48.896362+0000","last_became_peered":"2024-04-26T20:27:48.896362+0000","last_unstale":"2024-04-26T20:27:49.932303+0000","last_undegraded":"2024-04-26T20:27:49.932303+0000","last_fullsized":"2024-04-26T20:27:49.932303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924165+0000","last_change":"2024-04-26T20:27:49.920198+0000","last_active":"2024-04-26T20:27:49.924165+0000","last_peered":"2024-04-26T20:27:49.924165+0000","last_clean":"2024-04-26T20:27:49.924165+0000","last_became_active":"2024-04-26T20:27:49.919936+0000","last_became_peered":"2024-04-26T20:27:49.919936+0000","last_unstale":"2024-04-26T20:27:49.924165+0000","last_undegraded":"2024-04-26T20:27:49.924165+0000","last_fullsized":"2024-04-26T20:27:49.924165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918703+0000","last_change":"2024-04-26T20:27:48.891091+0000","last_active":"2024-04-26T20:27:49.918703+0000","last_peered":"2024-04-26T20:27:49.918703+0000","last_clean":"2024-04-26T20:27:49.918703+0000","last_became_active":"2024-04-26T20:27:48.888052+0000","last_became_peered":"2024-04-26T20:27:48.888052+0000","last_unstale":"2024-04-26T20:27:49.918703+0000","last_undegraded":"2024-04-26T20:27:49.918703+0000","last_fullsized":"2024-04-26T20:27:49.918703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:35.082176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.923837+0000","last_change":"2024-04-26T20:27:49.917196+0000","last_active":"2024-04-26T20:27:49.923837+0000","last_peered":"2024-04-26T20:27:49.923837+0000","last_clean":"2024-04-26T20:27:49.923837+0000","last_became_active":"2024-04-26T20:27:49.916941+0000","last_became_peered":"2024-04-26T20:27:49.916941+0000","last_unstale":"2024-04-26T20:27:49.923837+0000","last_undegraded":"2024-04-26T20:27:49.923837+0000","last_fullsized":"2024-04-26T20:27:49.923837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895346+0000","last_change":"2024-04-26T20:27:48.894730+0000","last_active":"2024-04-26T20:27:49.895346+0000","last_peered":"2024-04-26T20:27:49.895346+0000","last_clean":"2024-04-26T20:27:49.895346+0000","last_became_active":"2024-04-26T20:27:48.894446+0000","last_became_peered":"2024-04-26T20:27:48.894446+0000","last_unstale":"2024-04-26T20:27:49.895346+0000","last_undegraded":"2024-04-26T20:27:49.895346+0000","last_fullsized":"2024-04-26T20:27:49.895346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913576+0000","last_change":"2024-04-26T20:27:49.913576+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913576+0000","last_undegraded":"2024-04-26T20:27:49.913576+0000","last_fullsized":"2024-04-26T20:27:49.913576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904291+0000","last_change":"2024-04-26T20:27:48.886899+0000","last_active":"2024-04-26T20:27:49.904291+0000","last_peered":"2024-04-26T20:27:49.904291+0000","last_clean":"2024-04-26T20:27:49.904291+0000","last_became_active":"2024-04-26T20:27:48.886536+0000","last_became_peered":"2024-04-26T20:27:48.886536+0000","last_unstale":"2024-04-26T20:27:49.904291+0000","last_undegraded":"2024-04-26T20:27:49.904291+0000","last_fullsized":"2024-04-26T20:27:49.904291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929644+0000","last_change":"2024-04-26T20:27:49.918902+0000","last_active":"2024-04-26T20:27:49.929644+0000","last_peered":"2024-04-26T20:27:49.929644+0000","last_clean":"2024-04-26T20:27:49.929644+0000","last_became_active":"2024-04-26T20:27:49.918340+0000","last_became_peered":"2024-04-26T20:27:49.918340+0000","last_unstale":"2024-04-26T20:27:49.929644+0000","last_undegraded":"2024-04-26T20:27:49.929644+0000","last_fullsized":"2024-04-26T20:27:49.929644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917635+0000","last_change":"2024-04-26T20:27:48.886522+0000","last_active":"2024-04-26T20:27:49.917635+0000","last_peered":"2024-04-26T20:27:49.917635+0000","last_clean":"2024-04-26T20:27:49.917635+0000","last_became_active":"2024-04-26T20:27:48.885874+0000","last_became_peered":"2024-04-26T20:27:48.885874+0000","last_unstale":"2024-04-26T20:27:49.917635+0000","last_undegraded":"2024-04-26T20:27:49.917635+0000","last_fullsized":"2024-04-26T20:27:49.917635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932240+0000","last_change":"2024-04-26T20:27:49.926821+0000","last_active":"2024-04-26T20:27:49.932240+0000","last_peered":"2024-04-26T20:27:49.932240+0000","last_clean":"2024-04-26T20:27:49.932240+0000","last_became_active":"2024-04-26T20:27:49.926564+0000","last_became_peered":"2024-04-26T20:27:49.926564+0000","last_unstale":"2024-04-26T20:27:49.932240+0000","last_undegraded":"2024-04-26T20:27:49.932240+0000","last_fullsized":"2024-04-26T20:27:49.932240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924746+0000","last_change":"2024-04-26T20:27:48.900526+0000","last_active":"2024-04-26T20:27:49.924746+0000","last_peered":"2024-04-26T20:27:49.924746+0000","last_clean":"2024-04-26T20:27:49.924746+0000","last_became_active":"2024-04-26T20:27:48.900110+0000","last_became_peered":"2024-04-26T20:27:48.900110+0000","last_unstale":"2024-04-26T20:27:49.924746+0000","last_undegraded":"2024-04-26T20:27:49.924746+0000","last_fullsized":"2024-04-26T20:27:49.924746+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:26.369631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928774+0000","last_change":"2024-04-26T20:27:49.923959+0000","last_active":"2024-04-26T20:27:49.928774+0000","last_peered":"2024-04-26T20:27:49.928774+0000","last_clean":"2024-04-26T20:27:49.928774+0000","last_became_active":"2024-04-26T20:27:49.923064+0000","last_became_peered":"2024-04-26T20:27:49.923064+0000","last_unstale":"2024-04-26T20:27:49.928774+0000","last_undegraded":"2024-04-26T20:27:49.928774+0000","last_fullsized":"2024-04-26T20:27:49.928774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895794+0000","last_change":"2024-04-26T20:27:48.899650+0000","last_active":"2024-04-26T20:27:49.895794+0000","last_peered":"2024-04-26T20:27:49.895794+0000","last_clean":"2024-04-26T20:27:49.895794+0000","last_became_active":"2024-04-26T20:27:48.899036+0000","last_became_peered":"2024-04-26T20:27:48.899036+0000","last_unstale":"2024-04-26T20:27:49.895794+0000","last_undegraded":"2024-04-26T20:27:49.895794+0000","last_fullsized":"2024-04-26T20:27:49.895794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:33.237276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917237+0000","last_change":"2024-04-26T20:27:49.901218+0000","last_active":"2024-04-26T20:27:49.917237+0000","last_peered":"2024-04-26T20:27:49.917237+0000","last_clean":"2024-04-26T20:27:49.917237+0000","last_became_active":"2024-04-26T20:27:49.900732+0000","last_became_peered":"2024-04-26T20:27:49.900732+0000","last_unstale":"2024-04-26T20:27:49.917237+0000","last_undegraded":"2024-04-26T20:27:49.917237+0000","last_fullsized":"2024-04-26T20:27:49.917237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926905+0000","last_change":"2024-04-26T20:27:48.883412+0000","last_active":"2024-04-26T20:27:49.926905+0000","last_peered":"2024-04-26T20:27:49.926905+0000","last_clean":"2024-04-26T20:27:49.926905+0000","last_became_active":"2024-04-26T20:27:48.882939+0000","last_became_peered":"2024-04-26T20:27:48.882939+0000","last_unstale":"2024-04-26T20:27:49.926905+0000","last_undegraded":"2024-04-26T20:27:49.926905+0000","last_fullsized":"2024-04-26T20:27:49.926905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.174009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911670+0000","last_change":"2024-04-26T20:27:49.909503+0000","last_active":"2024-04-26T20:27:49.911670+0000","last_peered":"2024-04-26T20:27:49.911670+0000","last_clean":"2024-04-26T20:27:49.911670+0000","last_became_active":"2024-04-26T20:27:49.909122+0000","last_became_peered":"2024-04-26T20:27:49.909122+0000","last_unstale":"2024-04-26T20:27:49.911670+0000","last_undegraded":"2024-04-26T20:27:49.911670+0000","last_fullsized":"2024-04-26T20:27:49.911670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:46.024331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895256+0000","last_change":"2024-04-26T20:27:48.885014+0000","last_active":"2024-04-26T20:27:49.895256+0000","last_peered":"2024-04-26T20:27:49.895256+0000","last_clean":"2024-04-26T20:27:49.895256+0000","last_became_active":"2024-04-26T20:27:48.884727+0000","last_became_peered":"2024-04-26T20:27:48.884727+0000","last_unstale":"2024-04-26T20:27:49.895256+0000","last_undegraded":"2024-04-26T20:27:49.895256+0000","last_fullsized":"2024-04-26T20:27:49.895256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917288+0000","last_change":"2024-04-26T20:27:49.917288+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917288+0000","last_undegraded":"2024-04-26T20:27:49.917288+0000","last_fullsized":"2024-04-26T20:27:49.917288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916870+0000","last_change":"2024-04-26T20:27:48.880032+0000","last_active":"2024-04-26T20:27:49.916870+0000","last_peered":"2024-04-26T20:27:49.916870+0000","last_clean":"2024-04-26T20:27:49.916870+0000","last_became_active":"2024-04-26T20:27:48.878893+0000","last_became_peered":"2024-04-26T20:27:48.878893+0000","last_unstale":"2024-04-26T20:27:49.916870+0000","last_undegraded":"2024-04-26T20:27:49.916870+0000","last_fullsized":"2024-04-26T20:27:49.916870+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:04.775515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927225+0000","last_change":"2024-04-26T20:27:49.927225+0000","last_active":"2024-04-26T20:27:49.927225+0000","last_peered":"2024-04-26T20:27:49.927225+0000","last_clean":"2024-04-26T20:27:49.927225+0000","last_became_active":"2024-04-26T20:27:49.925074+0000","last_became_peered":"2024-04-26T20:27:49.925074+0000","last_unstale":"2024-04-26T20:27:49.927225+0000","last_undegraded":"2024-04-26T20:27:49.927225+0000","last_fullsized":"2024-04-26T20:27:49.927225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886368+0000","last_change":"2024-04-26T20:27:48.896812+0000","last_active":"2024-04-26T20:27:49.886368+0000","last_peered":"2024-04-26T20:27:49.886368+0000","last_clean":"2024-04-26T20:27:49.886368+0000","last_became_active":"2024-04-26T20:27:48.895863+0000","last_became_peered":"2024-04-26T20:27:48.895863+0000","last_unstale":"2024-04-26T20:27:49.886368+0000","last_undegraded":"2024-04-26T20:27:49.886368+0000","last_fullsized":"2024-04-26T20:27:49.886368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:43.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927088+0000","last_change":"2024-04-26T20:27:49.927088+0000","last_active":"2024-04-26T20:27:49.927088+0000","last_peered":"2024-04-26T20:27:49.927088+0000","last_clean":"2024-04-26T20:27:49.927088+0000","last_became_active":"2024-04-26T20:27:49.926795+0000","last_became_peered":"2024-04-26T20:27:49.926795+0000","last_unstale":"2024-04-26T20:27:49.927088+0000","last_undegraded":"2024-04-26T20:27:49.927088+0000","last_fullsized":"2024-04-26T20:27:49.927088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913876+0000","last_change":"2024-04-26T20:27:48.891475+0000","last_active":"2024-04-26T20:27:49.913876+0000","last_peered":"2024-04-26T20:27:49.913876+0000","last_clean":"2024-04-26T20:27:49.913876+0000","last_became_active":"2024-04-26T20:27:48.891058+0000","last_became_peered":"2024-04-26T20:27:48.891058+0000","last_unstale":"2024-04-26T20:27:49.913876+0000","last_undegraded":"2024-04-26T20:27:49.913876+0000","last_fullsized":"2024-04-26T20:27:49.913876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:23.923671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929667+0000","last_change":"2024-04-26T20:27:49.921492+0000","last_active":"2024-04-26T20:27:49.929667+0000","last_peered":"2024-04-26T20:27:49.929667+0000","last_clean":"2024-04-26T20:27:49.929667+0000","last_became_active":"2024-04-26T20:27:49.921208+0000","last_became_peered":"2024-04-26T20:27:49.921208+0000","last_unstale":"2024-04-26T20:27:49.929667+0000","last_undegraded":"2024-04-26T20:27:49.929667+0000","last_fullsized":"2024-04-26T20:27:49.929667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886522+0000","last_change":"2024-04-26T20:27:48.879281+0000","last_active":"2024-04-26T20:27:49.886522+0000","last_peered":"2024-04-26T20:27:49.886522+0000","last_clean":"2024-04-26T20:27:49.886522+0000","last_became_active":"2024-04-26T20:27:48.878964+0000","last_became_peered":"2024-04-26T20:27:48.878964+0000","last_unstale":"2024-04-26T20:27:49.886522+0000","last_undegraded":"2024-04-26T20:27:49.886522+0000","last_fullsized":"2024-04-26T20:27:49.886522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:32.854744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892290+0000","last_change":"2024-04-26T20:27:49.892290+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892290+0000","last_undegraded":"2024-04-26T20:27:49.892290+0000","last_fullsized":"2024-04-26T20:27:49.892290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924918+0000","last_change":"2024-04-26T20:27:48.896599+0000","last_active":"2024-04-26T20:27:49.924918+0000","last_peered":"2024-04-26T20:27:49.924918+0000","last_clean":"2024-04-26T20:27:49.924918+0000","last_became_active":"2024-04-26T20:27:48.896224+0000","last_became_peered":"2024-04-26T20:27:48.896224+0000","last_unstale":"2024-04-26T20:27:49.924918+0000","last_undegraded":"2024-04-26T20:27:49.924918+0000","last_fullsized":"2024-04-26T20:27:49.924918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:52.021385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.918360+0000","last_change":"2024-04-26T20:27:49.918360+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.918360+0000","last_undegraded":"2024-04-26T20:27:49.918360+0000","last_fullsized":"2024-04-26T20:27:49.918360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.887587+0000","last_change":"2024-04-26T20:27:48.892674+0000","last_active":"2024-04-26T20:27:49.887587+0000","last_peered":"2024-04-26T20:27:49.887587+0000","last_clean":"2024-04-26T20:27:49.887587+0000","last_became_active":"2024-04-26T20:27:48.892364+0000","last_became_peered":"2024-04-26T20:27:48.892364+0000","last_unstale":"2024-04-26T20:27:49.887587+0000","last_undegraded":"2024-04-26T20:27:49.887587+0000","last_fullsized":"2024-04-26T20:27:49.887587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:56.403870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925751+0000","last_change":"2024-04-26T20:27:49.918395+0000","last_active":"2024-04-26T20:27:49.925751+0000","last_peered":"2024-04-26T20:27:49.925751+0000","last_clean":"2024-04-26T20:27:49.925751+0000","last_became_active":"2024-04-26T20:27:49.918111+0000","last_became_peered":"2024-04-26T20:27:49.918111+0000","last_unstale":"2024-04-26T20:27:49.925751+0000","last_undegraded":"2024-04-26T20:27:49.925751+0000","last_fullsized":"2024-04-26T20:27:49.925751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.903165+0000","last_change":"2024-04-26T20:27:48.886260+0000","last_active":"2024-04-26T20:27:49.903165+0000","last_peered":"2024-04-26T20:27:49.903165+0000","last_clean":"2024-04-26T20:27:49.903165+0000","last_became_active":"2024-04-26T20:27:48.885935+0000","last_became_peered":"2024-04-26T20:27:48.885935+0000","last_unstale":"2024-04-26T20:27:49.903165+0000","last_undegraded":"2024-04-26T20:27:49.903165+0000","last_fullsized":"2024-04-26T20:27:49.903165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924077+0000","last_change":"2024-04-26T20:27:49.919086+0000","last_active":"2024-04-26T20:27:49.924077+0000","last_peered":"2024-04-26T20:27:49.924077+0000","last_clean":"2024-04-26T20:27:49.924077+0000","last_became_active":"2024-04-26T20:27:49.913716+0000","last_became_peered":"2024-04-26T20:27:49.913716+0000","last_unstale":"2024-04-26T20:27:49.924077+0000","last_undegraded":"2024-04-26T20:27:49.924077+0000","last_fullsized":"2024-04-26T20:27:49.924077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901017+0000","last_change":"2024-04-26T20:27:48.892408+0000","last_active":"2024-04-26T20:27:49.901017+0000","last_peered":"2024-04-26T20:27:49.901017+0000","last_clean":"2024-04-26T20:27:49.901017+0000","last_became_active":"2024-04-26T20:27:48.891998+0000","last_became_peered":"2024-04-26T20:27:48.891998+0000","last_unstale":"2024-04-26T20:27:49.901017+0000","last_undegraded":"2024-04-26T20:27:49.901017+0000","last_fullsized":"2024-04-26T20:27:49.901017+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.912958+0000","last_change":"2024-04-26T20:27:49.912958+0000","last_active":"2024-04-26T20:27:49.912958+0000","last_peered":"2024-04-26T20:27:49.912958+0000","last_clean":"2024-04-26T20:27:49.912958+0000","last_became_active":"2024-04-26T20:27:49.912656+0000","last_became_peered":"2024-04-26T20:27:49.912656+0000","last_unstale":"2024-04-26T20:27:49.912958+0000","last_undegraded":"2024-04-26T20:27:49.912958+0000","last_fullsized":"2024-04-26T20:27:49.912958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904813+0000","last_change":"2024-04-26T20:27:48.883806+0000","last_active":"2024-04-26T20:27:49.904813+0000","last_peered":"2024-04-26T20:27:49.904813+0000","last_clean":"2024-04-26T20:27:49.904813+0000","last_became_active":"2024-04-26T20:27:48.883495+0000","last_became_peered":"2024-04-26T20:27:48.883495+0000","last_unstale":"2024-04-26T20:27:49.904813+0000","last_undegraded":"2024-04-26T20:27:49.904813+0000","last_fullsized":"2024-04-26T20:27:49.904813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918725+0000","last_change":"2024-04-26T20:27:49.907230+0000","last_active":"2024-04-26T20:27:49.918725+0000","last_peered":"2024-04-26T20:27:49.918725+0000","last_clean":"2024-04-26T20:27:49.918725+0000","last_became_active":"2024-04-26T20:27:49.905877+0000","last_became_peered":"2024-04-26T20:27:49.905877+0000","last_unstale":"2024-04-26T20:27:49.918725+0000","last_undegraded":"2024-04-26T20:27:49.918725+0000","last_fullsized":"2024-04-26T20:27:49.918725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894422+0000","last_change":"2024-04-26T20:27:48.893801+0000","last_active":"2024-04-26T20:27:49.894422+0000","last_peered":"2024-04-26T20:27:49.894422+0000","last_clean":"2024-04-26T20:27:49.894422+0000","last_became_active":"2024-04-26T20:27:48.893424+0000","last_became_peered":"2024-04-26T20:27:48.893424+0000","last_unstale":"2024-04-26T20:27:49.894422+0000","last_undegraded":"2024-04-26T20:27:49.894422+0000","last_fullsized":"2024-04-26T20:27:49.894422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917924+0000","last_change":"2024-04-26T20:27:49.917924+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917924+0000","last_undegraded":"2024-04-26T20:27:49.917924+0000","last_fullsized":"2024-04-26T20:27:49.917924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.898659+0000","last_change":"2024-04-26T20:27:48.885205+0000","last_active":"2024-04-26T20:27:49.898659+0000","last_peered":"2024-04-26T20:27:49.898659+0000","last_clean":"2024-04-26T20:27:49.898659+0000","last_became_active":"2024-04-26T20:27:48.884908+0000","last_became_peered":"2024-04-26T20:27:48.884908+0000","last_unstale":"2024-04-26T20:27:49.898659+0000","last_undegraded":"2024-04-26T20:27:49.898659+0000","last_fullsized":"2024-04-26T20:27:49.898659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925449+0000","last_change":"2024-04-26T20:27:49.925449+0000","last_active":"2024-04-26T20:27:49.925449+0000","last_peered":"2024-04-26T20:27:49.925449+0000","last_clean":"2024-04-26T20:27:49.925449+0000","last_became_active":"2024-04-26T20:27:49.925098+0000","last_became_peered":"2024-04-26T20:27:49.925098+0000","last_unstale":"2024-04-26T20:27:49.925449+0000","last_undegraded":"2024-04-26T20:27:49.925449+0000","last_fullsized":"2024-04-26T20:27:49.925449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918901+0000","last_change":"2024-04-26T20:27:48.889736+0000","last_active":"2024-04-26T20:27:49.918901+0000","last_peered":"2024-04-26T20:27:49.918901+0000","last_clean":"2024-04-26T20:27:49.918901+0000","last_became_active":"2024-04-26T20:27:48.889103+0000","last_became_peered":"2024-04-26T20:27:48.889103+0000","last_unstale":"2024-04-26T20:27:49.918901+0000","last_undegraded":"2024-04-26T20:27:49.918901+0000","last_fullsized":"2024-04-26T20:27:49.918901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917420+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:27:49.917420+0000","last_peered":"2024-04-26T20:27:49.917420+0000","last_clean":"2024-04-26T20:27:49.917420+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:27:49.917420+0000","last_undegraded":"2024-04-26T20:27:49.917420+0000","last_fullsized":"2024-04-26T20:27:49.917420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":766739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:27:54.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:27:55.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.173+0000 7f1dd5600640 1 Processor -- start 2024-04-26T20:27:55.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.173+0000 7f1dd5600640 1 -- start start 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d1620 0x7f1dd00d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd5600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1dd00d3810 0x7f1dd00b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd5600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd00d0480 con 0x7f1dd00d1a00 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd5600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd00d0ce0 con 0x7f1dd00d3810 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd00d0720 con 0x7f1dd00d1620 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd4c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1dd00d3810 0x7f1dd00b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd4c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1dd00d3810 0x7f1dd00b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47410/0 (socket says 172.21.15.43:47410) 2024-04-26T20:27:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.175+0000 7f1dd4c00640 1 -- 172.21.15.43:0/2451745426 learned_addr learned my addr 172.21.15.43:0/2451745426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.176+0000 7f1dcfe00640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d1620 0x7f1dd00d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.176+0000 7f1dd4c00640 1 -- 172.21.15.43:0/2451745426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d1620 msgr2=0x7f1dd00d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.176+0000 7f1dd4c00640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d1620 0x7f1dd00d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.176+0000 7f1dd4c00640 1 -- 172.21.15.43:0/2451745426 >> v1:172.21.15.164:6789/0 conn(0x7f1dd00d1a00 legacy=0x7f1dd00b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.176+0000 7f1dd4c00640 1 -- 172.21.15.43:0/2451745426 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00d0a00 con 0x7f1dd00d3810 2024-04-26T20:27:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.177+0000 7f1dd4c00640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d3810 0x7f1dd00b5280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009e50 tx=0x7f1dc802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2907da70e138ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.177+0000 7f1dcf400640 1 -- 172.21.15.43:0/2451745426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8036070 con 0x7f1dd00d3810 2024-04-26T20:27:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dcf400640 1 -- 172.21.15.43:0/2451745426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8031600 con 0x7f1dd00d3810 2024-04-26T20:27:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dcf400640 1 -- 172.21.15.43:0/2451745426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80319d0 con 0x7f1dd00d3810 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 -- 172.21.15.43:0/2451745426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d3810 msgr2=0x7f1dd00b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d3810 0x7f1dd00b5280 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009e50 tx=0x7f1dc802a630 comp rx=0 tx=0).stop 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 -- 172.21.15.43:0/2451745426 shutdown_connections 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d3810 0x7f1dd00b5280 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 --2- 172.21.15.43:0/2451745426 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d1620 0x7f1dd00d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 -- 172.21.15.43:0/2451745426 >> 172.21.15.43:0/2451745426 conn(0x7f1dd00d4c30 msgr2=0x7f1dd00d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 -- 172.21.15.43:0/2451745426 shutdown_connections 2024-04-26T20:27:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.178+0000 7f1dd5600640 1 -- 172.21.15.43:0/2451745426 wait complete. 2024-04-26T20:27:55.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.181+0000 7f1dd5600640 1 Processor -- start 2024-04-26T20:27:55.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.181+0000 7f1dd5600640 1 -- start start 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd00d1620 0x7f1dd0146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 0x7f1dd00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00b57c0 con 0x7f1dd00d1a00 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00d1e60 con 0x7f1dd00d1620 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dd5600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd00b5940 con 0x7f1dd00d3810 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dc7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.182+0000 7f1dc7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47412/0 (socket says 172.21.15.43:47412) 2024-04-26T20:27:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 -- 172.21.15.43:0/4240108720 learned_addr learned my addr 172.21.15.43:0/4240108720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dcfe00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd00d1620 0x7f1dd0146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dd4c00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 0x7f1dd00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 -- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 msgr2=0x7f1dd00b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 0x7f1dd00b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 -- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd00d1620 msgr2=0x7f1dd0146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd00d1620 0x7f1dd0146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 -- 172.21.15.43:0/4240108720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00b9180 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dd4c00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 0x7f1dd00b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.183+0000 7f1dc7400640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c430 tx=0x7f1db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.184+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8011790 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.184+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.184+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd00b86e0 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.184+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8011bb0 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.184+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd00b9700 con 0x7f1dd00d1a00 2024-04-26T20:27:55.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.185+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd0069c20 con 0x7f1dd00d1a00 2024-04-26T20:27:55.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.186+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db801c460 con 0x7f1dd00d1a00 2024-04-26T20:27:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.187+0000 7f1dccc00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.187+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1db808bb90 con 0x7f1dd00d1a00 2024-04-26T20:27:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.187+0000 7f1dcfe00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.188+0000 7f1dcfe00640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 0x7f1dac06ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.192+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dd0069c20 con 0x7f1dd00d1a00 2024-04-26T20:27:55.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.392+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1dd00b9900 con 0x7f1dac06c920 2024-04-26T20:27:55.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.411+0000 7f1dccc00640 1 -- 172.21.15.43:0/4240108720 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250752 (secure 0 0 0) 0x7f1dd00b9900 con 0x7f1dac06c920 2024-04-26T20:27:55.414 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:55.418 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 msgr2=0x7f1dac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 0x7f1dac06ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0008040 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 msgr2=0x7f1dd01e9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c430 tx=0x7f1db800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 shutdown_connections 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1dac06c920 0x7f1dac06ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1dd00d3810 0x7f1dd00b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1dd00d1a00 0x7f1dd01e9bb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 --2- 172.21.15.43:0/4240108720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd00d1620 0x7f1dd0146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 >> 172.21.15.43:0/4240108720 conn(0x7f1dd00d4c30 msgr2=0x7f1dd006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.418+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 shutdown_connections 2024-04-26T20:27:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.419+0000 7f1dd5600640 1 -- 172.21.15.43:0/4240108720 wait complete. 2024-04-26T20:27:55.440 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-04-26T20:27:54.898966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591838,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237216,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737504,"statfs":{"total":773094113280,"available":772851204096,"internally_reserved":0,"allocated":4509696,"data_stored":2598871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.040038"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929111+0000","last_change":"2024-04-26T20:27:49.918830+0000","last_active":"2024-04-26T20:27:49.929111+0000","last_peered":"2024-04-26T20:27:49.929111+0000","last_clean":"2024-04-26T20:27:49.929111+0000","last_became_active":"2024-04-26T20:27:49.918530+0000","last_became_peered":"2024-04-26T20:27:49.918530+0000","last_unstale":"2024-04-26T20:27:49.929111+0000","last_undegraded":"2024-04-26T20:27:49.929111+0000","last_fullsized":"2024-04-26T20:27:49.929111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913015+0000","last_change":"2024-04-26T20:27:49.911220+0000","last_active":"2024-04-26T20:27:49.913015+0000","last_peered":"2024-04-26T20:27:49.913015+0000","last_clean":"2024-04-26T20:27:49.913015+0000","last_became_active":"2024-04-26T20:27:49.910949+0000","last_became_peered":"2024-04-26T20:27:49.910949+0000","last_unstale":"2024-04-26T20:27:49.913015+0000","last_undegraded":"2024-04-26T20:27:49.913015+0000","last_fullsized":"2024-04-26T20:27:49.913015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926512+0000","last_change":"2024-04-26T20:27:49.926512+0000","last_active":"2024-04-26T20:27:49.926512+0000","last_peered":"2024-04-26T20:27:49.926512+0000","last_clean":"2024-04-26T20:27:49.926512+0000","last_became_active":"2024-04-26T20:27:49.925884+0000","last_became_peered":"2024-04-26T20:27:49.925884+0000","last_unstale":"2024-04-26T20:27:49.926512+0000","last_undegraded":"2024-04-26T20:27:49.926512+0000","last_fullsized":"2024-04-26T20:27:49.926512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926623+0000","last_change":"2024-04-26T20:27:49.923178+0000","last_active":"2024-04-26T20:27:49.926623+0000","last_peered":"2024-04-26T20:27:49.926623+0000","last_clean":"2024-04-26T20:27:49.926623+0000","last_became_active":"2024-04-26T20:27:49.920812+0000","last_became_peered":"2024-04-26T20:27:49.920812+0000","last_unstale":"2024-04-26T20:27:49.926623+0000","last_undegraded":"2024-04-26T20:27:49.926623+0000","last_fullsized":"2024-04-26T20:27:49.926623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:40.526461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926391+0000","last_change":"2024-04-26T20:27:49.922091+0000","last_active":"2024-04-26T20:27:49.926391+0000","last_peered":"2024-04-26T20:27:49.926391+0000","last_clean":"2024-04-26T20:27:49.926391+0000","last_became_active":"2024-04-26T20:27:49.921810+0000","last_became_peered":"2024-04-26T20:27:49.921810+0000","last_unstale":"2024-04-26T20:27:49.926391+0000","last_undegraded":"2024-04-26T20:27:49.926391+0000","last_fullsized":"2024-04-26T20:27:49.926391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:13.735142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908694+0000","last_change":"2024-04-26T20:27:49.902670+0000","last_active":"2024-04-26T20:27:49.908694+0000","last_peered":"2024-04-26T20:27:49.908694+0000","last_clean":"2024-04-26T20:27:49.908694+0000","last_became_active":"2024-04-26T20:27:49.902336+0000","last_became_peered":"2024-04-26T20:27:49.902336+0000","last_unstale":"2024-04-26T20:27:49.908694+0000","last_undegraded":"2024-04-26T20:27:49.908694+0000","last_fullsized":"2024-04-26T20:27:49.908694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922231+0000","last_change":"2024-04-26T20:27:49.922231+0000","last_active":"2024-04-26T20:27:49.922231+0000","last_peered":"2024-04-26T20:27:49.922231+0000","last_clean":"2024-04-26T20:27:49.922231+0000","last_became_active":"2024-04-26T20:27:49.921927+0000","last_became_peered":"2024-04-26T20:27:49.921927+0000","last_unstale":"2024-04-26T20:27:49.922231+0000","last_undegraded":"2024-04-26T20:27:49.922231+0000","last_fullsized":"2024-04-26T20:27:49.922231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931179+0000","last_change":"2024-04-26T20:27:49.931179+0000","last_active":"2024-04-26T20:27:49.931179+0000","last_peered":"2024-04-26T20:27:49.931179+0000","last_clean":"2024-04-26T20:27:49.931179+0000","last_became_active":"2024-04-26T20:27:49.930033+0000","last_became_peered":"2024-04-26T20:27:49.930033+0000","last_unstale":"2024-04-26T20:27:49.931179+0000","last_undegraded":"2024-04-26T20:27:49.931179+0000","last_fullsized":"2024-04-26T20:27:49.931179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926173+0000","last_change":"2024-04-26T20:27:49.926173+0000","last_active":"2024-04-26T20:27:49.926173+0000","last_peered":"2024-04-26T20:27:49.926173+0000","last_clean":"2024-04-26T20:27:49.926173+0000","last_became_active":"2024-04-26T20:27:49.925901+0000","last_became_peered":"2024-04-26T20:27:49.925901+0000","last_unstale":"2024-04-26T20:27:49.926173+0000","last_undegraded":"2024-04-26T20:27:49.926173+0000","last_fullsized":"2024-04-26T20:27:49.926173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932243+0000","last_change":"2024-04-26T20:27:49.932243+0000","last_active":"2024-04-26T20:27:49.932243+0000","last_peered":"2024-04-26T20:27:49.932243+0000","last_clean":"2024-04-26T20:27:49.932243+0000","last_became_active":"2024-04-26T20:27:49.931924+0000","last_became_peered":"2024-04-26T20:27:49.931924+0000","last_unstale":"2024-04-26T20:27:49.932243+0000","last_undegraded":"2024-04-26T20:27:49.932243+0000","last_fullsized":"2024-04-26T20:27:49.932243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928800+0000","last_change":"2024-04-26T20:27:49.924453+0000","last_active":"2024-04-26T20:27:49.928800+0000","last_peered":"2024-04-26T20:27:49.928800+0000","last_clean":"2024-04-26T20:27:49.928800+0000","last_became_active":"2024-04-26T20:27:49.924193+0000","last_became_peered":"2024-04-26T20:27:49.924193+0000","last_unstale":"2024-04-26T20:27:49.928800+0000","last_undegraded":"2024-04-26T20:27:49.928800+0000","last_fullsized":"2024-04-26T20:27:49.928800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922276+0000","last_change":"2024-04-26T20:27:49.917930+0000","last_active":"2024-04-26T20:27:49.922276+0000","last_peered":"2024-04-26T20:27:49.922276+0000","last_clean":"2024-04-26T20:27:49.922276+0000","last_became_active":"2024-04-26T20:27:49.914720+0000","last_became_peered":"2024-04-26T20:27:49.914720+0000","last_unstale":"2024-04-26T20:27:49.922276+0000","last_undegraded":"2024-04-26T20:27:49.922276+0000","last_fullsized":"2024-04-26T20:27:49.922276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932579+0000","last_change":"2024-04-26T20:27:49.932579+0000","last_active":"2024-04-26T20:27:49.932579+0000","last_peered":"2024-04-26T20:27:49.932579+0000","last_clean":"2024-04-26T20:27:49.932579+0000","last_became_active":"2024-04-26T20:27:49.931863+0000","last_became_peered":"2024-04-26T20:27:49.931863+0000","last_unstale":"2024-04-26T20:27:49.932579+0000","last_undegraded":"2024-04-26T20:27:49.932579+0000","last_fullsized":"2024-04-26T20:27:49.932579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.891489+0000","last_change":"2024-04-26T20:27:49.891489+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.891489+0000","last_undegraded":"2024-04-26T20:27:49.891489+0000","last_fullsized":"2024-04-26T20:27:49.891489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929160+0000","last_change":"2024-04-26T20:27:49.929160+0000","last_active":"2024-04-26T20:27:49.929160+0000","last_peered":"2024-04-26T20:27:49.929160+0000","last_clean":"2024-04-26T20:27:49.929160+0000","last_became_active":"2024-04-26T20:27:49.928881+0000","last_became_peered":"2024-04-26T20:27:49.928881+0000","last_unstale":"2024-04-26T20:27:49.929160+0000","last_undegraded":"2024-04-26T20:27:49.929160+0000","last_fullsized":"2024-04-26T20:27:49.929160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913270+0000","last_change":"2024-04-26T20:27:49.913270+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913270+0000","last_undegraded":"2024-04-26T20:27:49.913270+0000","last_fullsized":"2024-04-26T20:27:49.913270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.921031+0000","last_change":"2024-04-26T20:27:49.921031+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.921031+0000","last_undegraded":"2024-04-26T20:27:49.921031+0000","last_fullsized":"2024-04-26T20:27:49.921031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927012+0000","last_change":"2024-04-26T20:27:49.927012+0000","last_active":"2024-04-26T20:27:49.927012+0000","last_peered":"2024-04-26T20:27:49.927012+0000","last_clean":"2024-04-26T20:27:49.927012+0000","last_became_active":"2024-04-26T20:27:49.926696+0000","last_became_peered":"2024-04-26T20:27:49.926696+0000","last_unstale":"2024-04-26T20:27:49.927012+0000","last_undegraded":"2024-04-26T20:27:49.927012+0000","last_fullsized":"2024-04-26T20:27:49.927012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911696+0000","last_change":"2024-04-26T20:27:49.902695+0000","last_active":"2024-04-26T20:27:49.911696+0000","last_peered":"2024-04-26T20:27:49.911696+0000","last_clean":"2024-04-26T20:27:49.911696+0000","last_became_active":"2024-04-26T20:27:49.901838+0000","last_became_peered":"2024-04-26T20:27:49.901838+0000","last_unstale":"2024-04-26T20:27:49.911696+0000","last_undegraded":"2024-04-26T20:27:49.911696+0000","last_fullsized":"2024-04-26T20:27:49.911696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:44.890223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926250+0000","last_change":"2024-04-26T20:27:49.926250+0000","last_active":"2024-04-26T20:27:49.926250+0000","last_peered":"2024-04-26T20:27:49.926250+0000","last_clean":"2024-04-26T20:27:49.926250+0000","last_became_active":"2024-04-26T20:27:49.925945+0000","last_became_peered":"2024-04-26T20:27:49.925945+0000","last_unstale":"2024-04-26T20:27:49.926250+0000","last_undegraded":"2024-04-26T20:27:49.926250+0000","last_fullsized":"2024-04-26T20:27:49.926250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928074+0000","last_change":"2024-04-26T20:27:49.928074+0000","last_active":"2024-04-26T20:27:49.928074+0000","last_peered":"2024-04-26T20:27:49.928074+0000","last_clean":"2024-04-26T20:27:49.928074+0000","last_became_active":"2024-04-26T20:27:49.920120+0000","last_became_peered":"2024-04-26T20:27:49.920120+0000","last_unstale":"2024-04-26T20:27:49.928074+0000","last_undegraded":"2024-04-26T20:27:49.928074+0000","last_fullsized":"2024-04-26T20:27:49.928074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916486+0000","last_change":"2024-04-26T20:27:49.906288+0000","last_active":"2024-04-26T20:27:49.916486+0000","last_peered":"2024-04-26T20:27:49.916486+0000","last_clean":"2024-04-26T20:27:49.916486+0000","last_became_active":"2024-04-26T20:27:49.905307+0000","last_became_peered":"2024-04-26T20:27:49.905307+0000","last_unstale":"2024-04-26T20:27:49.916486+0000","last_undegraded":"2024-04-26T20:27:49.916486+0000","last_fullsized":"2024-04-26T20:27:49.916486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913198+0000","last_change":"2024-04-26T20:27:49.913198+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913198+0000","last_undegraded":"2024-04-26T20:27:49.913198+0000","last_fullsized":"2024-04-26T20:27:49.913198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925927+0000","last_change":"2024-04-26T20:27:49.921300+0000","last_active":"2024-04-26T20:27:49.925927+0000","last_peered":"2024-04-26T20:27:49.925927+0000","last_clean":"2024-04-26T20:27:49.925927+0000","last_became_active":"2024-04-26T20:27:49.921025+0000","last_became_peered":"2024-04-26T20:27:49.921025+0000","last_unstale":"2024-04-26T20:27:49.925927+0000","last_undegraded":"2024-04-26T20:27:49.925927+0000","last_fullsized":"2024-04-26T20:27:49.925927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:04.556567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917029+0000","last_change":"2024-04-26T20:27:49.917029+0000","last_active":"2024-04-26T20:27:49.917029+0000","last_peered":"2024-04-26T20:27:49.917029+0000","last_clean":"2024-04-26T20:27:49.917029+0000","last_became_active":"2024-04-26T20:27:49.916684+0000","last_became_peered":"2024-04-26T20:27:49.916684+0000","last_unstale":"2024-04-26T20:27:49.917029+0000","last_undegraded":"2024-04-26T20:27:49.917029+0000","last_fullsized":"2024-04-26T20:27:49.917029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.893547+0000","last_change":"2024-04-26T20:27:49.893547+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.893547+0000","last_undegraded":"2024-04-26T20:27:49.893547+0000","last_fullsized":"2024-04-26T20:27:49.893547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.915570+0000","last_change":"2024-04-26T20:27:49.915570+0000","last_active":"2024-04-26T20:27:49.915570+0000","last_peered":"2024-04-26T20:27:49.915570+0000","last_clean":"2024-04-26T20:27:49.915570+0000","last_became_active":"2024-04-26T20:27:49.915277+0000","last_became_peered":"2024-04-26T20:27:49.915277+0000","last_unstale":"2024-04-26T20:27:49.915570+0000","last_undegraded":"2024-04-26T20:27:49.915570+0000","last_fullsized":"2024-04-26T20:27:49.915570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929083+0000","last_change":"2024-04-26T20:27:49.923260+0000","last_active":"2024-04-26T20:27:49.929083+0000","last_peered":"2024-04-26T20:27:49.929083+0000","last_clean":"2024-04-26T20:27:49.929083+0000","last_became_active":"2024-04-26T20:27:49.922900+0000","last_became_peered":"2024-04-26T20:27:49.922900+0000","last_unstale":"2024-04-26T20:27:49.929083+0000","last_undegraded":"2024-04-26T20:27:49.929083+0000","last_fullsized":"2024-04-26T20:27:49.929083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918298+0000","last_change":"2024-04-26T20:27:49.918298+0000","last_active":"2024-04-26T20:27:49.918298+0000","last_peered":"2024-04-26T20:27:49.918298+0000","last_clean":"2024-04-26T20:27:49.918298+0000","last_became_active":"2024-04-26T20:27:49.912487+0000","last_became_peered":"2024-04-26T20:27:49.912487+0000","last_unstale":"2024-04-26T20:27:49.918298+0000","last_undegraded":"2024-04-26T20:27:49.918298+0000","last_fullsized":"2024-04-26T20:27:49.918298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925689+0000","last_change":"2024-04-26T20:27:49.914198+0000","last_active":"2024-04-26T20:27:49.925689+0000","last_peered":"2024-04-26T20:27:49.925689+0000","last_clean":"2024-04-26T20:27:49.925689+0000","last_became_active":"2024-04-26T20:27:49.913905+0000","last_became_peered":"2024-04-26T20:27:49.913905+0000","last_unstale":"2024-04-26T20:27:49.925689+0000","last_undegraded":"2024-04-26T20:27:49.925689+0000","last_fullsized":"2024-04-26T20:27:49.925689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:34.250292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913586+0000","last_change":"2024-04-26T20:27:49.913586+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913586+0000","last_undegraded":"2024-04-26T20:27:49.913586+0000","last_fullsized":"2024-04-26T20:27:49.913586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908366+0000","last_change":"2024-04-26T20:27:49.899934+0000","last_active":"2024-04-26T20:27:49.908366+0000","last_peered":"2024-04-26T20:27:49.908366+0000","last_clean":"2024-04-26T20:27:49.908366+0000","last_became_active":"2024-04-26T20:27:49.899653+0000","last_became_peered":"2024-04-26T20:27:49.899653+0000","last_unstale":"2024-04-26T20:27:49.908366+0000","last_undegraded":"2024-04-26T20:27:49.908366+0000","last_fullsized":"2024-04-26T20:27:49.908366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.919146+0000","last_change":"2024-04-26T20:27:49.919146+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.919146+0000","last_undegraded":"2024-04-26T20:27:49.919146+0000","last_fullsized":"2024-04-26T20:27:49.919146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911517+0000","last_change":"2024-04-26T20:27:48.897659+0000","last_active":"2024-04-26T20:27:49.911517+0000","last_peered":"2024-04-26T20:27:49.911517+0000","last_clean":"2024-04-26T20:27:49.911517+0000","last_became_active":"2024-04-26T20:27:48.896450+0000","last_became_peered":"2024-04-26T20:27:48.896450+0000","last_unstale":"2024-04-26T20:27:49.911517+0000","last_undegraded":"2024-04-26T20:27:49.911517+0000","last_fullsized":"2024-04-26T20:27:49.911517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892621+0000","last_change":"2024-04-26T20:27:49.892621+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892621+0000","last_undegraded":"2024-04-26T20:27:49.892621+0000","last_fullsized":"2024-04-26T20:27:49.892621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901581+0000","last_change":"2024-04-26T20:27:48.895441+0000","last_active":"2024-04-26T20:27:49.901581+0000","last_peered":"2024-04-26T20:27:49.901581+0000","last_clean":"2024-04-26T20:27:49.901581+0000","last_became_active":"2024-04-26T20:27:48.895118+0000","last_became_peered":"2024-04-26T20:27:48.895118+0000","last_unstale":"2024-04-26T20:27:49.901581+0000","last_undegraded":"2024-04-26T20:27:49.901581+0000","last_fullsized":"2024-04-26T20:27:49.901581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.890220+0000","last_change":"2024-04-26T20:27:49.890220+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.890220+0000","last_undegraded":"2024-04-26T20:27:49.890220+0000","last_fullsized":"2024-04-26T20:27:49.890220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895459+0000","last_change":"2024-04-26T20:27:48.894724+0000","last_active":"2024-04-26T20:27:49.895459+0000","last_peered":"2024-04-26T20:27:49.895459+0000","last_clean":"2024-04-26T20:27:49.895459+0000","last_became_active":"2024-04-26T20:27:48.893381+0000","last_became_peered":"2024-04-26T20:27:48.893381+0000","last_unstale":"2024-04-26T20:27:49.895459+0000","last_undegraded":"2024-04-26T20:27:49.895459+0000","last_fullsized":"2024-04-26T20:27:49.895459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931161+0000","last_change":"2024-04-26T20:27:49.929224+0000","last_active":"2024-04-26T20:27:49.931161+0000","last_peered":"2024-04-26T20:27:49.931161+0000","last_clean":"2024-04-26T20:27:49.931161+0000","last_became_active":"2024-04-26T20:27:49.928934+0000","last_became_peered":"2024-04-26T20:27:49.928934+0000","last_unstale":"2024-04-26T20:27:49.931161+0000","last_undegraded":"2024-04-26T20:27:49.931161+0000","last_fullsized":"2024-04-26T20:27:49.931161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895394+0000","last_change":"2024-04-26T20:27:48.894846+0000","last_active":"2024-04-26T20:27:49.895394+0000","last_peered":"2024-04-26T20:27:49.895394+0000","last_clean":"2024-04-26T20:27:49.895394+0000","last_became_active":"2024-04-26T20:27:48.893698+0000","last_became_peered":"2024-04-26T20:27:48.893698+0000","last_unstale":"2024-04-26T20:27:49.895394+0000","last_undegraded":"2024-04-26T20:27:49.895394+0000","last_fullsized":"2024-04-26T20:27:49.895394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925175+0000","last_change":"2024-04-26T20:27:49.920847+0000","last_active":"2024-04-26T20:27:49.925175+0000","last_peered":"2024-04-26T20:27:49.925175+0000","last_clean":"2024-04-26T20:27:49.925175+0000","last_became_active":"2024-04-26T20:27:49.920522+0000","last_became_peered":"2024-04-26T20:27:49.920522+0000","last_unstale":"2024-04-26T20:27:49.925175+0000","last_undegraded":"2024-04-26T20:27:49.925175+0000","last_fullsized":"2024-04-26T20:27:49.925175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.930867+0000","last_change":"2024-04-26T20:27:48.891209+0000","last_active":"2024-04-26T20:27:49.930867+0000","last_peered":"2024-04-26T20:27:49.930867+0000","last_clean":"2024-04-26T20:27:49.930867+0000","last_became_active":"2024-04-26T20:27:48.890894+0000","last_became_peered":"2024-04-26T20:27:48.890894+0000","last_unstale":"2024-04-26T20:27:49.930867+0000","last_undegraded":"2024-04-26T20:27:49.930867+0000","last_fullsized":"2024-04-26T20:27:49.930867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928684+0000","last_change":"2024-04-26T20:27:49.920288+0000","last_active":"2024-04-26T20:27:49.928684+0000","last_peered":"2024-04-26T20:27:49.928684+0000","last_clean":"2024-04-26T20:27:49.928684+0000","last_became_active":"2024-04-26T20:27:49.919780+0000","last_became_peered":"2024-04-26T20:27:49.919780+0000","last_unstale":"2024-04-26T20:27:49.928684+0000","last_undegraded":"2024-04-26T20:27:49.928684+0000","last_fullsized":"2024-04-26T20:27:49.928684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922040+0000","last_change":"2024-04-26T20:27:48.897588+0000","last_active":"2024-04-26T20:27:49.922040+0000","last_peered":"2024-04-26T20:27:49.922040+0000","last_clean":"2024-04-26T20:27:49.922040+0000","last_became_active":"2024-04-26T20:27:48.896574+0000","last_became_peered":"2024-04-26T20:27:48.896574+0000","last_unstale":"2024-04-26T20:27:49.922040+0000","last_undegraded":"2024-04-26T20:27:49.922040+0000","last_fullsized":"2024-04-26T20:27:49.922040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:22.623531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.919624+0000","last_change":"2024-04-26T20:27:49.916775+0000","last_active":"2024-04-26T20:27:49.919624+0000","last_peered":"2024-04-26T20:27:49.919624+0000","last_clean":"2024-04-26T20:27:49.919624+0000","last_became_active":"2024-04-26T20:27:49.916312+0000","last_became_peered":"2024-04-26T20:27:49.916312+0000","last_unstale":"2024-04-26T20:27:49.919624+0000","last_undegraded":"2024-04-26T20:27:49.919624+0000","last_fullsized":"2024-04-26T20:27:49.919624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895250+0000","last_change":"2024-04-26T20:27:48.898227+0000","last_active":"2024-04-26T20:27:49.895250+0000","last_peered":"2024-04-26T20:27:49.895250+0000","last_clean":"2024-04-26T20:27:49.895250+0000","last_became_active":"2024-04-26T20:27:48.897196+0000","last_became_peered":"2024-04-26T20:27:48.897196+0000","last_unstale":"2024-04-26T20:27:49.895250+0000","last_undegraded":"2024-04-26T20:27:49.895250+0000","last_fullsized":"2024-04-26T20:27:49.895250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:46.834398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.900144+0000","last_change":"2024-04-26T20:27:49.900144+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.900144+0000","last_undegraded":"2024-04-26T20:27:49.900144+0000","last_fullsized":"2024-04-26T20:27:49.900144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894039+0000","last_change":"2024-04-26T20:27:48.890830+0000","last_active":"2024-04-26T20:27:49.894039+0000","last_peered":"2024-04-26T20:27:49.894039+0000","last_clean":"2024-04-26T20:27:49.894039+0000","last_became_active":"2024-04-26T20:27:48.890544+0000","last_became_peered":"2024-04-26T20:27:48.890544+0000","last_unstale":"2024-04-26T20:27:49.894039+0000","last_undegraded":"2024-04-26T20:27:49.894039+0000","last_fullsized":"2024-04-26T20:27:49.894039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927244+0000","last_change":"2024-04-26T20:27:49.923399+0000","last_active":"2024-04-26T20:27:49.927244+0000","last_peered":"2024-04-26T20:27:49.927244+0000","last_clean":"2024-04-26T20:27:49.927244+0000","last_became_active":"2024-04-26T20:27:49.923127+0000","last_became_peered":"2024-04-26T20:27:49.923127+0000","last_unstale":"2024-04-26T20:27:49.927244+0000","last_undegraded":"2024-04-26T20:27:49.927244+0000","last_fullsized":"2024-04-26T20:27:49.927244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:24.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901840+0000","last_change":"2024-04-26T20:27:48.889003+0000","last_active":"2024-04-26T20:27:49.901840+0000","last_peered":"2024-04-26T20:27:49.901840+0000","last_clean":"2024-04-26T20:27:49.901840+0000","last_became_active":"2024-04-26T20:27:48.888708+0000","last_became_peered":"2024-04-26T20:27:48.888708+0000","last_unstale":"2024-04-26T20:27:49.901840+0000","last_undegraded":"2024-04-26T20:27:49.901840+0000","last_fullsized":"2024-04-26T20:27:49.901840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:17.802067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917978+0000","last_change":"2024-04-26T20:27:49.917978+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917978+0000","last_undegraded":"2024-04-26T20:27:49.917978+0000","last_fullsized":"2024-04-26T20:27:49.917978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922887+0000","last_change":"2024-04-26T20:27:48.897655+0000","last_active":"2024-04-26T20:27:49.922887+0000","last_peered":"2024-04-26T20:27:49.922887+0000","last_clean":"2024-04-26T20:27:49.922887+0000","last_became_active":"2024-04-26T20:27:48.897232+0000","last_became_peered":"2024-04-26T20:27:48.897232+0000","last_unstale":"2024-04-26T20:27:49.922887+0000","last_undegraded":"2024-04-26T20:27:49.922887+0000","last_fullsized":"2024-04-26T20:27:49.922887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925441+0000","last_change":"2024-04-26T20:27:49.921057+0000","last_active":"2024-04-26T20:27:49.925441+0000","last_peered":"2024-04-26T20:27:49.925441+0000","last_clean":"2024-04-26T20:27:49.925441+0000","last_became_active":"2024-04-26T20:27:49.920747+0000","last_became_peered":"2024-04-26T20:27:49.920747+0000","last_unstale":"2024-04-26T20:27:49.925441+0000","last_undegraded":"2024-04-26T20:27:49.925441+0000","last_fullsized":"2024-04-26T20:27:49.925441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:40.387985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922912+0000","last_change":"2024-04-26T20:27:48.895154+0000","last_active":"2024-04-26T20:27:49.922912+0000","last_peered":"2024-04-26T20:27:49.922912+0000","last_clean":"2024-04-26T20:27:49.922912+0000","last_became_active":"2024-04-26T20:27:48.894300+0000","last_became_peered":"2024-04-26T20:27:48.894300+0000","last_unstale":"2024-04-26T20:27:49.922912+0000","last_undegraded":"2024-04-26T20:27:49.922912+0000","last_fullsized":"2024-04-26T20:27:49.922912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.920045+0000","last_change":"2024-04-26T20:27:49.920045+0000","last_active":"2024-04-26T20:27:49.920045+0000","last_peered":"2024-04-26T20:27:49.920045+0000","last_clean":"2024-04-26T20:27:49.920045+0000","last_became_active":"2024-04-26T20:27:49.919739+0000","last_became_peered":"2024-04-26T20:27:49.919739+0000","last_unstale":"2024-04-26T20:27:49.920045+0000","last_undegraded":"2024-04-26T20:27:49.920045+0000","last_fullsized":"2024-04-26T20:27:49.920045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932303+0000","last_change":"2024-04-26T20:27:48.896675+0000","last_active":"2024-04-26T20:27:49.932303+0000","last_peered":"2024-04-26T20:27:49.932303+0000","last_clean":"2024-04-26T20:27:49.932303+0000","last_became_active":"2024-04-26T20:27:48.896362+0000","last_became_peered":"2024-04-26T20:27:48.896362+0000","last_unstale":"2024-04-26T20:27:49.932303+0000","last_undegraded":"2024-04-26T20:27:49.932303+0000","last_fullsized":"2024-04-26T20:27:49.932303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924165+0000","last_change":"2024-04-26T20:27:49.920198+0000","last_active":"2024-04-26T20:27:49.924165+0000","last_peered":"2024-04-26T20:27:49.924165+0000","last_clean":"2024-04-26T20:27:49.924165+0000","last_became_active":"2024-04-26T20:27:49.919936+0000","last_became_peered":"2024-04-26T20:27:49.919936+0000","last_unstale":"2024-04-26T20:27:49.924165+0000","last_undegraded":"2024-04-26T20:27:49.924165+0000","last_fullsized":"2024-04-26T20:27:49.924165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918703+0000","last_change":"2024-04-26T20:27:48.891091+0000","last_active":"2024-04-26T20:27:49.918703+0000","last_peered":"2024-04-26T20:27:49.918703+0000","last_clean":"2024-04-26T20:27:49.918703+0000","last_became_active":"2024-04-26T20:27:48.888052+0000","last_became_peered":"2024-04-26T20:27:48.888052+0000","last_unstale":"2024-04-26T20:27:49.918703+0000","last_undegraded":"2024-04-26T20:27:49.918703+0000","last_fullsized":"2024-04-26T20:27:49.918703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:35.082176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.923837+0000","last_change":"2024-04-26T20:27:49.917196+0000","last_active":"2024-04-26T20:27:49.923837+0000","last_peered":"2024-04-26T20:27:49.923837+0000","last_clean":"2024-04-26T20:27:49.923837+0000","last_became_active":"2024-04-26T20:27:49.916941+0000","last_became_peered":"2024-04-26T20:27:49.916941+0000","last_unstale":"2024-04-26T20:27:49.923837+0000","last_undegraded":"2024-04-26T20:27:49.923837+0000","last_fullsized":"2024-04-26T20:27:49.923837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895346+0000","last_change":"2024-04-26T20:27:48.894730+0000","last_active":"2024-04-26T20:27:49.895346+0000","last_peered":"2024-04-26T20:27:49.895346+0000","last_clean":"2024-04-26T20:27:49.895346+0000","last_became_active":"2024-04-26T20:27:48.894446+0000","last_became_peered":"2024-04-26T20:27:48.894446+0000","last_unstale":"2024-04-26T20:27:49.895346+0000","last_undegraded":"2024-04-26T20:27:49.895346+0000","last_fullsized":"2024-04-26T20:27:49.895346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913576+0000","last_change":"2024-04-26T20:27:49.913576+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913576+0000","last_undegraded":"2024-04-26T20:27:49.913576+0000","last_fullsized":"2024-04-26T20:27:49.913576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904291+0000","last_change":"2024-04-26T20:27:48.886899+0000","last_active":"2024-04-26T20:27:49.904291+0000","last_peered":"2024-04-26T20:27:49.904291+0000","last_clean":"2024-04-26T20:27:49.904291+0000","last_became_active":"2024-04-26T20:27:48.886536+0000","last_became_peered":"2024-04-26T20:27:48.886536+0000","last_unstale":"2024-04-26T20:27:49.904291+0000","last_undegraded":"2024-04-26T20:27:49.904291+0000","last_fullsized":"2024-04-26T20:27:49.904291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929644+0000","last_change":"2024-04-26T20:27:49.918902+0000","last_active":"2024-04-26T20:27:49.929644+0000","last_peered":"2024-04-26T20:27:49.929644+0000","last_clean":"2024-04-26T20:27:49.929644+0000","last_became_active":"2024-04-26T20:27:49.918340+0000","last_became_peered":"2024-04-26T20:27:49.918340+0000","last_unstale":"2024-04-26T20:27:49.929644+0000","last_undegraded":"2024-04-26T20:27:49.929644+0000","last_fullsized":"2024-04-26T20:27:49.929644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917635+0000","last_change":"2024-04-26T20:27:48.886522+0000","last_active":"2024-04-26T20:27:49.917635+0000","last_peered":"2024-04-26T20:27:49.917635+0000","last_clean":"2024-04-26T20:27:49.917635+0000","last_became_active":"2024-04-26T20:27:48.885874+0000","last_became_peered":"2024-04-26T20:27:48.885874+0000","last_unstale":"2024-04-26T20:27:49.917635+0000","last_undegraded":"2024-04-26T20:27:49.917635+0000","last_fullsized":"2024-04-26T20:27:49.917635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932240+0000","last_change":"2024-04-26T20:27:49.926821+0000","last_active":"2024-04-26T20:27:49.932240+0000","last_peered":"2024-04-26T20:27:49.932240+0000","last_clean":"2024-04-26T20:27:49.932240+0000","last_became_active":"2024-04-26T20:27:49.926564+0000","last_became_peered":"2024-04-26T20:27:49.926564+0000","last_unstale":"2024-04-26T20:27:49.932240+0000","last_undegraded":"2024-04-26T20:27:49.932240+0000","last_fullsized":"2024-04-26T20:27:49.932240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924746+0000","last_change":"2024-04-26T20:27:48.900526+0000","last_active":"2024-04-26T20:27:49.924746+0000","last_peered":"2024-04-26T20:27:49.924746+0000","last_clean":"2024-04-26T20:27:49.924746+0000","last_became_active":"2024-04-26T20:27:48.900110+0000","last_became_peered":"2024-04-26T20:27:48.900110+0000","last_unstale":"2024-04-26T20:27:49.924746+0000","last_undegraded":"2024-04-26T20:27:49.924746+0000","last_fullsized":"2024-04-26T20:27:49.924746+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:26.369631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928774+0000","last_change":"2024-04-26T20:27:49.923959+0000","last_active":"2024-04-26T20:27:49.928774+0000","last_peered":"2024-04-26T20:27:49.928774+0000","last_clean":"2024-04-26T20:27:49.928774+0000","last_became_active":"2024-04-26T20:27:49.923064+0000","last_became_peered":"2024-04-26T20:27:49.923064+0000","last_unstale":"2024-04-26T20:27:49.928774+0000","last_undegraded":"2024-04-26T20:27:49.928774+0000","last_fullsized":"2024-04-26T20:27:49.928774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895794+0000","last_change":"2024-04-26T20:27:48.899650+0000","last_active":"2024-04-26T20:27:49.895794+0000","last_peered":"2024-04-26T20:27:49.895794+0000","last_clean":"2024-04-26T20:27:49.895794+0000","last_became_active":"2024-04-26T20:27:48.899036+0000","last_became_peered":"2024-04-26T20:27:48.899036+0000","last_unstale":"2024-04-26T20:27:49.895794+0000","last_undegraded":"2024-04-26T20:27:49.895794+0000","last_fullsized":"2024-04-26T20:27:49.895794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:33.237276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917237+0000","last_change":"2024-04-26T20:27:49.901218+0000","last_active":"2024-04-26T20:27:49.917237+0000","last_peered":"2024-04-26T20:27:49.917237+0000","last_clean":"2024-04-26T20:27:49.917237+0000","last_became_active":"2024-04-26T20:27:49.900732+0000","last_became_peered":"2024-04-26T20:27:49.900732+0000","last_unstale":"2024-04-26T20:27:49.917237+0000","last_undegraded":"2024-04-26T20:27:49.917237+0000","last_fullsized":"2024-04-26T20:27:49.917237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926905+0000","last_change":"2024-04-26T20:27:48.883412+0000","last_active":"2024-04-26T20:27:49.926905+0000","last_peered":"2024-04-26T20:27:49.926905+0000","last_clean":"2024-04-26T20:27:49.926905+0000","last_became_active":"2024-04-26T20:27:48.882939+0000","last_became_peered":"2024-04-26T20:27:48.882939+0000","last_unstale":"2024-04-26T20:27:49.926905+0000","last_undegraded":"2024-04-26T20:27:49.926905+0000","last_fullsized":"2024-04-26T20:27:49.926905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.174009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911670+0000","last_change":"2024-04-26T20:27:49.909503+0000","last_active":"2024-04-26T20:27:49.911670+0000","last_peered":"2024-04-26T20:27:49.911670+0000","last_clean":"2024-04-26T20:27:49.911670+0000","last_became_active":"2024-04-26T20:27:49.909122+0000","last_became_peered":"2024-04-26T20:27:49.909122+0000","last_unstale":"2024-04-26T20:27:49.911670+0000","last_undegraded":"2024-04-26T20:27:49.911670+0000","last_fullsized":"2024-04-26T20:27:49.911670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:46.024331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895256+0000","last_change":"2024-04-26T20:27:48.885014+0000","last_active":"2024-04-26T20:27:49.895256+0000","last_peered":"2024-04-26T20:27:49.895256+0000","last_clean":"2024-04-26T20:27:49.895256+0000","last_became_active":"2024-04-26T20:27:48.884727+0000","last_became_peered":"2024-04-26T20:27:48.884727+0000","last_unstale":"2024-04-26T20:27:49.895256+0000","last_undegraded":"2024-04-26T20:27:49.895256+0000","last_fullsized":"2024-04-26T20:27:49.895256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917288+0000","last_change":"2024-04-26T20:27:49.917288+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917288+0000","last_undegraded":"2024-04-26T20:27:49.917288+0000","last_fullsized":"2024-04-26T20:27:49.917288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916870+0000","last_change":"2024-04-26T20:27:48.880032+0000","last_active":"2024-04-26T20:27:49.916870+0000","last_peered":"2024-04-26T20:27:49.916870+0000","last_clean":"2024-04-26T20:27:49.916870+0000","last_became_active":"2024-04-26T20:27:48.878893+0000","last_became_peered":"2024-04-26T20:27:48.878893+0000","last_unstale":"2024-04-26T20:27:49.916870+0000","last_undegraded":"2024-04-26T20:27:49.916870+0000","last_fullsized":"2024-04-26T20:27:49.916870+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:04.775515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927225+0000","last_change":"2024-04-26T20:27:49.927225+0000","last_active":"2024-04-26T20:27:49.927225+0000","last_peered":"2024-04-26T20:27:49.927225+0000","last_clean":"2024-04-26T20:27:49.927225+0000","last_became_active":"2024-04-26T20:27:49.925074+0000","last_became_peered":"2024-04-26T20:27:49.925074+0000","last_unstale":"2024-04-26T20:27:49.927225+0000","last_undegraded":"2024-04-26T20:27:49.927225+0000","last_fullsized":"2024-04-26T20:27:49.927225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886368+0000","last_change":"2024-04-26T20:27:48.896812+0000","last_active":"2024-04-26T20:27:49.886368+0000","last_peered":"2024-04-26T20:27:49.886368+0000","last_clean":"2024-04-26T20:27:49.886368+0000","last_became_active":"2024-04-26T20:27:48.895863+0000","last_became_peered":"2024-04-26T20:27:48.895863+0000","last_unstale":"2024-04-26T20:27:49.886368+0000","last_undegraded":"2024-04-26T20:27:49.886368+0000","last_fullsized":"2024-04-26T20:27:49.886368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:43.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927088+0000","last_change":"2024-04-26T20:27:49.927088+0000","last_active":"2024-04-26T20:27:49.927088+0000","last_peered":"2024-04-26T20:27:49.927088+0000","last_clean":"2024-04-26T20:27:49.927088+0000","last_became_active":"2024-04-26T20:27:49.926795+0000","last_became_peered":"2024-04-26T20:27:49.926795+0000","last_unstale":"2024-04-26T20:27:49.927088+0000","last_undegraded":"2024-04-26T20:27:49.927088+0000","last_fullsized":"2024-04-26T20:27:49.927088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913876+0000","last_change":"2024-04-26T20:27:48.891475+0000","last_active":"2024-04-26T20:27:49.913876+0000","last_peered":"2024-04-26T20:27:49.913876+0000","last_clean":"2024-04-26T20:27:49.913876+0000","last_became_active":"2024-04-26T20:27:48.891058+0000","last_became_peered":"2024-04-26T20:27:48.891058+0000","last_unstale":"2024-04-26T20:27:49.913876+0000","last_undegraded":"2024-04-26T20:27:49.913876+0000","last_fullsized":"2024-04-26T20:27:49.913876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:23.923671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929667+0000","last_change":"2024-04-26T20:27:49.921492+0000","last_active":"2024-04-26T20:27:49.929667+0000","last_peered":"2024-04-26T20:27:49.929667+0000","last_clean":"2024-04-26T20:27:49.929667+0000","last_became_active":"2024-04-26T20:27:49.921208+0000","last_became_peered":"2024-04-26T20:27:49.921208+0000","last_unstale":"2024-04-26T20:27:49.929667+0000","last_undegraded":"2024-04-26T20:27:49.929667+0000","last_fullsized":"2024-04-26T20:27:49.929667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886522+0000","last_change":"2024-04-26T20:27:48.879281+0000","last_active":"2024-04-26T20:27:49.886522+0000","last_peered":"2024-04-26T20:27:49.886522+0000","last_clean":"2024-04-26T20:27:49.886522+0000","last_became_active":"2024-04-26T20:27:48.878964+0000","last_became_peered":"2024-04-26T20:27:48.878964+0000","last_unstale":"2024-04-26T20:27:49.886522+0000","last_undegraded":"2024-04-26T20:27:49.886522+0000","last_fullsized":"2024-04-26T20:27:49.886522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:32.854744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892290+0000","last_change":"2024-04-26T20:27:49.892290+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892290+0000","last_undegraded":"2024-04-26T20:27:49.892290+0000","last_fullsized":"2024-04-26T20:27:49.892290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924918+0000","last_change":"2024-04-26T20:27:48.896599+0000","last_active":"2024-04-26T20:27:49.924918+0000","last_peered":"2024-04-26T20:27:49.924918+0000","last_clean":"2024-04-26T20:27:49.924918+0000","last_became_active":"2024-04-26T20:27:48.896224+0000","last_became_peered":"2024-04-26T20:27:48.896224+0000","last_unstale":"2024-04-26T20:27:49.924918+0000","last_undegraded":"2024-04-26T20:27:49.924918+0000","last_fullsized":"2024-04-26T20:27:49.924918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:52.021385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.918360+0000","last_change":"2024-04-26T20:27:49.918360+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.918360+0000","last_undegraded":"2024-04-26T20:27:49.918360+0000","last_fullsized":"2024-04-26T20:27:49.918360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.887587+0000","last_change":"2024-04-26T20:27:48.892674+0000","last_active":"2024-04-26T20:27:49.887587+0000","last_peered":"2024-04-26T20:27:49.887587+0000","last_clean":"2024-04-26T20:27:49.887587+0000","last_became_active":"2024-04-26T20:27:48.892364+0000","last_became_peered":"2024-04-26T20:27:48.892364+0000","last_unstale":"2024-04-26T20:27:49.887587+0000","last_undegraded":"2024-04-26T20:27:49.887587+0000","last_fullsized":"2024-04-26T20:27:49.887587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:56.403870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925751+0000","last_change":"2024-04-26T20:27:49.918395+0000","last_active":"2024-04-26T20:27:49.925751+0000","last_peered":"2024-04-26T20:27:49.925751+0000","last_clean":"2024-04-26T20:27:49.925751+0000","last_became_active":"2024-04-26T20:27:49.918111+0000","last_became_peered":"2024-04-26T20:27:49.918111+0000","last_unstale":"2024-04-26T20:27:49.925751+0000","last_undegraded":"2024-04-26T20:27:49.925751+0000","last_fullsized":"2024-04-26T20:27:49.925751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.903165+0000","last_change":"2024-04-26T20:27:48.886260+0000","last_active":"2024-04-26T20:27:49.903165+0000","last_peered":"2024-04-26T20:27:49.903165+0000","last_clean":"2024-04-26T20:27:49.903165+0000","last_became_active":"2024-04-26T20:27:48.885935+0000","last_became_peered":"2024-04-26T20:27:48.885935+0000","last_unstale":"2024-04-26T20:27:49.903165+0000","last_undegraded":"2024-04-26T20:27:49.903165+0000","last_fullsized":"2024-04-26T20:27:49.903165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924077+0000","last_change":"2024-04-26T20:27:49.919086+0000","last_active":"2024-04-26T20:27:49.924077+0000","last_peered":"2024-04-26T20:27:49.924077+0000","last_clean":"2024-04-26T20:27:49.924077+0000","last_became_active":"2024-04-26T20:27:49.913716+0000","last_became_peered":"2024-04-26T20:27:49.913716+0000","last_unstale":"2024-04-26T20:27:49.924077+0000","last_undegraded":"2024-04-26T20:27:49.924077+0000","last_fullsized":"2024-04-26T20:27:49.924077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901017+0000","last_change":"2024-04-26T20:27:48.892408+0000","last_active":"2024-04-26T20:27:49.901017+0000","last_peered":"2024-04-26T20:27:49.901017+0000","last_clean":"2024-04-26T20:27:49.901017+0000","last_became_active":"2024-04-26T20:27:48.891998+0000","last_became_peered":"2024-04-26T20:27:48.891998+0000","last_unstale":"2024-04-26T20:27:49.901017+0000","last_undegraded":"2024-04-26T20:27:49.901017+0000","last_fullsized":"2024-04-26T20:27:49.901017+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.912958+0000","last_change":"2024-04-26T20:27:49.912958+0000","last_active":"2024-04-26T20:27:49.912958+0000","last_peered":"2024-04-26T20:27:49.912958+0000","last_clean":"2024-04-26T20:27:49.912958+0000","last_became_active":"2024-04-26T20:27:49.912656+0000","last_became_peered":"2024-04-26T20:27:49.912656+0000","last_unstale":"2024-04-26T20:27:49.912958+0000","last_undegraded":"2024-04-26T20:27:49.912958+0000","last_fullsized":"2024-04-26T20:27:49.912958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904813+0000","last_change":"2024-04-26T20:27:48.883806+0000","last_active":"2024-04-26T20:27:49.904813+0000","last_peered":"2024-04-26T20:27:49.904813+0000","last_clean":"2024-04-26T20:27:49.904813+0000","last_became_active":"2024-04-26T20:27:48.883495+0000","last_became_peered":"2024-04-26T20:27:48.883495+0000","last_unstale":"2024-04-26T20:27:49.904813+0000","last_undegraded":"2024-04-26T20:27:49.904813+0000","last_fullsized":"2024-04-26T20:27:49.904813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918725+0000","last_change":"2024-04-26T20:27:49.907230+0000","last_active":"2024-04-26T20:27:49.918725+0000","last_peered":"2024-04-26T20:27:49.918725+0000","last_clean":"2024-04-26T20:27:49.918725+0000","last_became_active":"2024-04-26T20:27:49.905877+0000","last_became_peered":"2024-04-26T20:27:49.905877+0000","last_unstale":"2024-04-26T20:27:49.918725+0000","last_undegraded":"2024-04-26T20:27:49.918725+0000","last_fullsized":"2024-04-26T20:27:49.918725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894422+0000","last_change":"2024-04-26T20:27:48.893801+0000","last_active":"2024-04-26T20:27:49.894422+0000","last_peered":"2024-04-26T20:27:49.894422+0000","last_clean":"2024-04-26T20:27:49.894422+0000","last_became_active":"2024-04-26T20:27:48.893424+0000","last_became_peered":"2024-04-26T20:27:48.893424+0000","last_unstale":"2024-04-26T20:27:49.894422+0000","last_undegraded":"2024-04-26T20:27:49.894422+0000","last_fullsized":"2024-04-26T20:27:49.894422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917924+0000","last_change":"2024-04-26T20:27:49.917924+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917924+0000","last_undegraded":"2024-04-26T20:27:49.917924+0000","last_fullsized":"2024-04-26T20:27:49.917924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.898659+0000","last_change":"2024-04-26T20:27:48.885205+0000","last_active":"2024-04-26T20:27:49.898659+0000","last_peered":"2024-04-26T20:27:49.898659+0000","last_clean":"2024-04-26T20:27:49.898659+0000","last_became_active":"2024-04-26T20:27:48.884908+0000","last_became_peered":"2024-04-26T20:27:48.884908+0000","last_unstale":"2024-04-26T20:27:49.898659+0000","last_undegraded":"2024-04-26T20:27:49.898659+0000","last_fullsized":"2024-04-26T20:27:49.898659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925449+0000","last_change":"2024-04-26T20:27:49.925449+0000","last_active":"2024-04-26T20:27:49.925449+0000","last_peered":"2024-04-26T20:27:49.925449+0000","last_clean":"2024-04-26T20:27:49.925449+0000","last_became_active":"2024-04-26T20:27:49.925098+0000","last_became_peered":"2024-04-26T20:27:49.925098+0000","last_unstale":"2024-04-26T20:27:49.925449+0000","last_undegraded":"2024-04-26T20:27:49.925449+0000","last_fullsized":"2024-04-26T20:27:49.925449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918901+0000","last_change":"2024-04-26T20:27:48.889736+0000","last_active":"2024-04-26T20:27:49.918901+0000","last_peered":"2024-04-26T20:27:49.918901+0000","last_clean":"2024-04-26T20:27:49.918901+0000","last_became_active":"2024-04-26T20:27:48.889103+0000","last_became_peered":"2024-04-26T20:27:48.889103+0000","last_unstale":"2024-04-26T20:27:49.918901+0000","last_undegraded":"2024-04-26T20:27:49.918901+0000","last_fullsized":"2024-04-26T20:27:49.918901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917420+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:27:49.917420+0000","last_peered":"2024-04-26T20:27:49.917420+0000","last_clean":"2024-04-26T20:27:49.917420+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:27:49.917420+0000","last_undegraded":"2024-04-26T20:27:49.917420+0000","last_fullsized":"2024-04-26T20:27:49.917420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":766739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.891489+0000', 'last_change': '2024-04-26T20:27:49.891489+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.891489+0000', 'last_undegraded': '2024-04-26T20:27:49.891489+0000', 'last_fullsized': '2024-04-26T20:27:49.891489+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.913270+0000', 'last_change': '2024-04-26T20:27:49.913270+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.913270+0000', 'last_undegraded': '2024-04-26T20:27:49.913270+0000', 'last_fullsized': '2024-04-26T20:27:49.913270+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-26T20:27:55.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.921031+0000', 'last_change': '2024-04-26T20:27:49.921031+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.921031+0000', 'last_undegraded': '2024-04-26T20:27:49.921031+0000', 'last_fullsized': '2024-04-26T20:27:49.921031+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.913198+0000', 'last_change': '2024-04-26T20:27:49.913198+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.913198+0000', 'last_undegraded': '2024-04-26T20:27:49.913198+0000', 'last_fullsized': '2024-04-26T20:27:49.913198+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.893547+0000', 'last_change': '2024-04-26T20:27:49.893547+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.893547+0000', 'last_undegraded': '2024-04-26T20:27:49.893547+0000', 'last_fullsized': '2024-04-26T20:27:49.893547+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.913586+0000', 'last_change': '2024-04-26T20:27:49.913586+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.913586+0000', 'last_undegraded': '2024-04-26T20:27:49.913586+0000', 'last_fullsized': '2024-04-26T20:27:49.913586+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.919146+0000', 'last_change': '2024-04-26T20:27:49.919146+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.919146+0000', 'last_undegraded': '2024-04-26T20:27:49.919146+0000', 'last_fullsized': '2024-04-26T20:27:49.919146+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-26T20:27:55.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.892621+0000', 'last_change': '2024-04-26T20:27:49.892621+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.892621+0000', 'last_undegraded': '2024-04-26T20:27:49.892621+0000', 'last_fullsized': '2024-04-26T20:27:49.892621+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.890220+0000', 'last_change': '2024-04-26T20:27:49.890220+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.890220+0000', 'last_undegraded': '2024-04-26T20:27:49.890220+0000', 'last_fullsized': '2024-04-26T20:27:49.890220+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.900144+0000', 'last_change': '2024-04-26T20:27:49.900144+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.900144+0000', 'last_undegraded': '2024-04-26T20:27:49.900144+0000', 'last_fullsized': '2024-04-26T20:27:49.900144+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': ['3', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.917978+0000', 'last_change': '2024-04-26T20:27:49.917978+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.917978+0000', 'last_undegraded': '2024-04-26T20:27:49.917978+0000', 'last_fullsized': '2024-04-26T20:27:49.917978+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.913576+0000', 'last_change': '2024-04-26T20:27:49.913576+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.913576+0000', 'last_undegraded': '2024-04-26T20:27:49.913576+0000', 'last_fullsized': '2024-04-26T20:27:49.913576+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-26T20:27:55.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.917288+0000', 'last_change': '2024-04-26T20:27:49.917288+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.917288+0000', 'last_undegraded': '2024-04-26T20:27:49.917288+0000', 'last_fullsized': '2024-04-26T20:27:49.917288+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.892290+0000', 'last_change': '2024-04-26T20:27:49.892290+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.892290+0000', 'last_undegraded': '2024-04-26T20:27:49.892290+0000', 'last_fullsized': '2024-04-26T20:27:49.892290+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.918360+0000', 'last_change': '2024-04-26T20:27:49.918360+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.918360+0000', 'last_undegraded': '2024-04-26T20:27:49.918360+0000', 'last_fullsized': '2024-04-26T20:27:49.918360+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-26T20:27:55.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-26T20:27:49.917924+0000', 'last_change': '2024-04-26T20:27:49.917924+0000', 'last_active': '2024-04-26T20:27:48.849301+0000', 'last_peered': '2024-04-26T20:27:48.849301+0000', 'last_clean': '2024-04-26T20:27:48.849301+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:27:49.917924+0000', 'last_undegraded': '2024-04-26T20:27:49.917924+0000', 'last_fullsized': '2024-04-26T20:27:49.917924+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'last_clean_scrub_stamp': '2024-04-26T20:27:48.849301+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:27:55.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:27:55.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.694+0000 7fedd7200640 1 Processor -- start 2024-04-26T20:27:55.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.694+0000 7fedd7200640 1 -- start start 2024-04-26T20:27:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd7200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedd0161460 0x7fedd0160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd7200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd00b50e0 con 0x7fedd01463e0 2024-04-26T20:27:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd7200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd00b4560 con 0x7fedd0160d90 2024-04-26T20:27:55.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd7200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fedd01611d0 con 0x7fedd0161460 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd5400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedd0161460 0x7fedd0160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd5400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedd0161460 0x7fedd0160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47428/0 (socket says 172.21.15.43:47428) 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.696+0000 7fedd5400640 1 -- 172.21.15.43:0/3109038746 learned_addr learned my addr 172.21.15.43:0/3109038746 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.697+0000 7fedd5400640 1 -- 172.21.15.43:0/3109038746 >> v1:172.21.15.164:6789/0 conn(0x7fedd0160d90 legacy=0x7fedd006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.697+0000 7fedd5400640 1 -- 172.21.15.43:0/3109038746 >> v1:172.21.15.43:6789/0 conn(0x7fedd01463e0 legacy=0x7fedd00b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.697+0000 7fedd5400640 1 -- 172.21.15.43:0/3109038746 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00b4840 con 0x7fedd0161460 2024-04-26T20:27:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.698+0000 7fedd5400640 1 --2- 172.21.15.43:0/3109038746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0161460 0x7fedd0160850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98d7d1d09efd37f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.698+0000 7fedd4a00640 1 -- 172.21.15.43:0/3109038746 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0037070 con 0x7fedd0161460 2024-04-26T20:27:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.698+0000 7fedd4a00640 1 -- 172.21.15.43:0/3109038746 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc0032440 con 0x7fedd0161460 2024-04-26T20:27:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.698+0000 7fedd4a00640 1 -- 172.21.15.43:0/3109038746 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0032810 con 0x7fedd0161460 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 -- 172.21.15.43:0/3109038746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0161460 msgr2=0x7fedd0160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 --2- 172.21.15.43:0/3109038746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0161460 0x7fedd0160850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc002a210 comp rx=0 tx=0).stop 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 -- 172.21.15.43:0/3109038746 shutdown_connections 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 --2- 172.21.15.43:0/3109038746 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0161460 0x7fedd0160850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 -- 172.21.15.43:0/3109038746 >> 172.21.15.43:0/3109038746 conn(0x7fedd00b26b0 msgr2=0x7fedd00b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:55.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 -- 172.21.15.43:0/3109038746 shutdown_connections 2024-04-26T20:27:55.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.699+0000 7fedd7200640 1 -- 172.21.15.43:0/3109038746 wait complete. 2024-04-26T20:27:55.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.701+0000 7fedd7200640 1 Processor -- start 2024-04-26T20:27:55.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.701+0000 7fedd7200640 1 -- start start 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 0x7fedd013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 0x7fedd013d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01467c0 con 0x7fedd0160d90 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00b4e00 con 0x7fedd0161460 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd7200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fedd006e760 con 0x7fedd01463e0 2024-04-26T20:27:55.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd5400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47432/0 (socket says 172.21.15.43:47432) 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.702+0000 7fedd5400640 1 -- 172.21.15.43:0/212460907 learned_addr learned my addr 172.21.15.43:0/212460907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd6800640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 0x7fedd013d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5e00640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 0x7fedd013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 -- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 msgr2=0x7fedd013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 0x7fedd013c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 -- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 msgr2=0x7fedd013d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 0x7fedd013d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 -- 172.21.15.43:0/212460907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd015e640 con 0x7fedd0160d90 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd6800640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 0x7fedd013d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5e00640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 0x7fedd013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd5400640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0041020 con 0x7fedd0160d90 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc0030970 con 0x7fedd0160d90 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0037040 con 0x7fedd0160d90 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.703+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd013c8f0 con 0x7fedd0160d90 2024-04-26T20:27:55.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.704+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd015eb90 con 0x7fedd0160d90 2024-04-26T20:27:55.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.705+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0069350 con 0x7fedd0160d90 2024-04-26T20:27:55.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.706+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedc0044020 con 0x7fedd0160d90 2024-04-26T20:27:55.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.707+0000 7fedca000640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 0x7fedb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:55.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.707+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fedc00ac420 con 0x7fedd0160d90 2024-04-26T20:27:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.707+0000 7fedd5e00640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 0x7fedb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.708+0000 7fedd5e00640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 0x7fedb406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fedc4001680 tx=0x7fedc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:55.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.711+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fedc004b070 con 0x7fedd0160d90 2024-04-26T20:27:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.993+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fedd00cfdf0 con 0x7fedd0160d90 2024-04-26T20:27:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.993+0000 7fedca000640 1 -- 172.21.15.43:0/212460907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1102 (secure 0 0 0) 0x7fedc007a620 con 0x7fedd0160d90 2024-04-26T20:27:55.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:55.995 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1714163149,"num_in_osds":8,"osd_in_since":1714163140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":81},{"state_name":"creating+activating","count":16}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":591838,"bytes_used":242909184,"bytes_avail":772851204096,"bytes_total":773094113280,"inactive_pgs_ratio":0.16494844853878021,"write_bytes_sec":2203,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5520}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-26T20:27:00.880955+0000","services":{}},"progress_events":{}} 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 msgr2=0x7fedb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 0x7fedb406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fedc4001680 tx=0x7fedc4009040 comp rx=0 tx=0).stop 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 msgr2=0x7fedd0144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc00046c0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 shutdown_connections 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fedb406c8f0 0x7fedb406edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd0161460 0x7fedd013d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedd0160d90 0x7fedd0144230 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 --2- 172.21.15.43:0/212460907 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd01463e0 0x7fedd013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.996+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 >> 172.21.15.43:0/212460907 conn(0x7fedd00b26b0 msgr2=0x7fedd00cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.997+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 shutdown_connections 2024-04-26T20:27:55.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:55.997+0000 7fedd7200640 1 -- 172.21.15.43:0/212460907 wait complete. 2024-04-26T20:27:56.018 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:27:56.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:27:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.231+0000 7f7579600640 1 Processor -- start 2024-04-26T20:27:56.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.231+0000 7f7579600640 1 -- start start 2024-04-26T20:27:56.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.233+0000 7f7579600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.233+0000 7f7579600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f75740d1060 0x7f757406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.233+0000 7f7579600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7574146ba0 con 0x7f75741467c0 2024-04-26T20:27:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.233+0000 7f7579600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7574073dd0 con 0x7f75740d1060 2024-04-26T20:27:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.233+0000 7f7579600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f75740731c0 con 0x7f7574072b70 2024-04-26T20:27:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.234+0000 7f7572000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.234+0000 7f7572000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:45898/0 (socket says 172.21.15.43:45898) 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.234+0000 7f7572000640 1 -- 172.21.15.43:0/3977107073 learned_addr learned my addr 172.21.15.43:0/3977107073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.234+0000 7f7572a00640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f75741467c0 0x7f75740b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:55638/0 (socket says 172.21.15.43:55638) 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.234+0000 7f7573400640 1 --2- 172.21.15.43:0/3977107073 >> v2:172.21.15.164:3300/0 conn(0x7f75740d1060 0x7f757406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.235+0000 7f7572000640 1 -- 172.21.15.43:0/3977107073 >> v2:172.21.15.164:3300/0 conn(0x7f75740d1060 msgr2=0x7f757406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.235+0000 7f7572000640 1 --2- 172.21.15.43:0/3977107073 >> v2:172.21.15.164:3300/0 conn(0x7f75740d1060 0x7f757406dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.235+0000 7f7572000640 1 -- 172.21.15.43:0/3977107073 >> v1:172.21.15.164:6789/0 conn(0x7f75741467c0 legacy=0x7f75740b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.235+0000 7f7572000640 1 -- 172.21.15.43:0/3977107073 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740734a0 con 0x7f7574072b70 2024-04-26T20:27:56.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.236+0000 7f7572000640 1 --2- 172.21.15.43:0/3977107073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7574073f50 tx=0x7f755c02a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dceb0822a11db637 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7571600640 1 -- 172.21.15.43:0/3977107073 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c036070 con 0x7f7574072b70 2024-04-26T20:27:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7571600640 1 -- 172.21.15.43:0/3977107073 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c031600 con 0x7f7574072b70 2024-04-26T20:27:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7571600640 1 -- 172.21.15.43:0/3977107073 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0319d0 con 0x7f7574072b70 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 -- 172.21.15.43:0/3977107073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 msgr2=0x7f75740d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 --2- 172.21.15.43:0/3977107073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7574073f50 tx=0x7f755c02a610 comp rx=0 tx=0).stop 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 -- 172.21.15.43:0/3977107073 shutdown_connections 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 --2- 172.21.15.43:0/3977107073 >> v2:172.21.15.164:3300/0 conn(0x7f75740d1060 0x7f757406dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 --2- 172.21.15.43:0/3977107073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75740d0b20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.237+0000 7f7579600640 1 -- 172.21.15.43:0/3977107073 >> 172.21.15.43:0/3977107073 conn(0x7f75740b2900 msgr2=0x7f75740b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.238+0000 7f7579600640 1 -- 172.21.15.43:0/3977107073 shutdown_connections 2024-04-26T20:27:56.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.238+0000 7f7579600640 1 -- 172.21.15.43:0/3977107073 wait complete. 2024-04-26T20:27:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.239+0000 7f7579600640 1 Processor -- start 2024-04-26T20:27:56.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 -- start start 2024-04-26T20:27:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75741f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75741467c0 0x7f75741f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f757406e530 con 0x7f75740d1060 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7574073a60 con 0x7f75741467c0 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.240+0000 7f7579600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f757406e6b0 con 0x7f7574072b70 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36656/0 (socket says 172.21.15.43:36656) 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 -- 172.21.15.43:0/544318300 learned_addr learned my addr 172.21.15.43:0/544318300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572a00640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75741f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7573400640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75741467c0 0x7f75741f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 -- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 msgr2=0x7f75741f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75741f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 -- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75741467c0 msgr2=0x7f75741f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75741467c0 0x7f75741f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.241+0000 7f7572000640 1 -- 172.21.15.43:0/544318300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75741f5380 con 0x7f75740d1060 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f7572000640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f755c009f10 tx=0x7f755c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c036040 con 0x7f75740d1060 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c003990 con 0x7f75740d1060 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75741f5660 con 0x7f75740d1060 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c003d80 con 0x7f75740d1060 2024-04-26T20:27:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.242+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75741f5c20 con 0x7f75740d1060 2024-04-26T20:27:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.244+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7574069350 con 0x7f75740d1060 2024-04-26T20:27:56.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.245+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f755c0087e0 con 0x7f75740d1060 2024-04-26T20:27:56.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.246+0000 7f756aa00640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 0x7f755006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:56.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.246+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f755c0acee0 con 0x7f75740d1060 2024-04-26T20:27:56.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.246+0000 7f7572a00640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 0x7f755006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:56.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.247+0000 7f7572a00640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 0x7f755006ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7564000f60 tx=0x7f7564007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:56.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.250+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f755c04b070 con 0x7f75740d1060 2024-04-26T20:27:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.454+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75740b25e0 con 0x7f755006c940 2024-04-26T20:27:56.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.472+0000 7f756aa00640 1 -- 172.21.15.43:0/544318300 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250752 (secure 0 0 0) 0x7f75740b25e0 con 0x7f755006c940 2024-04-26T20:27:56.478 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:56.479 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 msgr2=0x7f755006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 0x7f755006ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7564000f60 tx=0x7f7564007c60 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 msgr2=0x7f7574141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f755c009f10 tx=0x7f755c004a40 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 shutdown_connections 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f755006c940 0x7f755006ee00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75741467c0 0x7f75741f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75740d1060 0x7f7574141460 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 --2- 172.21.15.43:0/544318300 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7574072b70 0x7f75741f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 >> 172.21.15.43:0/544318300 conn(0x7f75740b2900 msgr2=0x7f75740afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 shutdown_connections 2024-04-26T20:27:56.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:56.478+0000 7f7579600640 1 -- 172.21.15.43:0/544318300 wait complete. 2024-04-26T20:27:56.500 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-04-26T20:27:54.898966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591838,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237216,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737504,"statfs":{"total":773094113280,"available":772851204096,"internally_reserved":0,"allocated":4509696,"data_stored":2598871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.040038"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929111+0000","last_change":"2024-04-26T20:27:49.918830+0000","last_active":"2024-04-26T20:27:49.929111+0000","last_peered":"2024-04-26T20:27:49.929111+0000","last_clean":"2024-04-26T20:27:49.929111+0000","last_became_active":"2024-04-26T20:27:49.918530+0000","last_became_peered":"2024-04-26T20:27:49.918530+0000","last_unstale":"2024-04-26T20:27:49.929111+0000","last_undegraded":"2024-04-26T20:27:49.929111+0000","last_fullsized":"2024-04-26T20:27:49.929111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913015+0000","last_change":"2024-04-26T20:27:49.911220+0000","last_active":"2024-04-26T20:27:49.913015+0000","last_peered":"2024-04-26T20:27:49.913015+0000","last_clean":"2024-04-26T20:27:49.913015+0000","last_became_active":"2024-04-26T20:27:49.910949+0000","last_became_peered":"2024-04-26T20:27:49.910949+0000","last_unstale":"2024-04-26T20:27:49.913015+0000","last_undegraded":"2024-04-26T20:27:49.913015+0000","last_fullsized":"2024-04-26T20:27:49.913015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926512+0000","last_change":"2024-04-26T20:27:49.926512+0000","last_active":"2024-04-26T20:27:49.926512+0000","last_peered":"2024-04-26T20:27:49.926512+0000","last_clean":"2024-04-26T20:27:49.926512+0000","last_became_active":"2024-04-26T20:27:49.925884+0000","last_became_peered":"2024-04-26T20:27:49.925884+0000","last_unstale":"2024-04-26T20:27:49.926512+0000","last_undegraded":"2024-04-26T20:27:49.926512+0000","last_fullsized":"2024-04-26T20:27:49.926512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926623+0000","last_change":"2024-04-26T20:27:49.923178+0000","last_active":"2024-04-26T20:27:49.926623+0000","last_peered":"2024-04-26T20:27:49.926623+0000","last_clean":"2024-04-26T20:27:49.926623+0000","last_became_active":"2024-04-26T20:27:49.920812+0000","last_became_peered":"2024-04-26T20:27:49.920812+0000","last_unstale":"2024-04-26T20:27:49.926623+0000","last_undegraded":"2024-04-26T20:27:49.926623+0000","last_fullsized":"2024-04-26T20:27:49.926623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:40.526461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926391+0000","last_change":"2024-04-26T20:27:49.922091+0000","last_active":"2024-04-26T20:27:49.926391+0000","last_peered":"2024-04-26T20:27:49.926391+0000","last_clean":"2024-04-26T20:27:49.926391+0000","last_became_active":"2024-04-26T20:27:49.921810+0000","last_became_peered":"2024-04-26T20:27:49.921810+0000","last_unstale":"2024-04-26T20:27:49.926391+0000","last_undegraded":"2024-04-26T20:27:49.926391+0000","last_fullsized":"2024-04-26T20:27:49.926391+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:13.735142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908694+0000","last_change":"2024-04-26T20:27:49.902670+0000","last_active":"2024-04-26T20:27:49.908694+0000","last_peered":"2024-04-26T20:27:49.908694+0000","last_clean":"2024-04-26T20:27:49.908694+0000","last_became_active":"2024-04-26T20:27:49.902336+0000","last_became_peered":"2024-04-26T20:27:49.902336+0000","last_unstale":"2024-04-26T20:27:49.908694+0000","last_undegraded":"2024-04-26T20:27:49.908694+0000","last_fullsized":"2024-04-26T20:27:49.908694+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922231+0000","last_change":"2024-04-26T20:27:49.922231+0000","last_active":"2024-04-26T20:27:49.922231+0000","last_peered":"2024-04-26T20:27:49.922231+0000","last_clean":"2024-04-26T20:27:49.922231+0000","last_became_active":"2024-04-26T20:27:49.921927+0000","last_became_peered":"2024-04-26T20:27:49.921927+0000","last_unstale":"2024-04-26T20:27:49.922231+0000","last_undegraded":"2024-04-26T20:27:49.922231+0000","last_fullsized":"2024-04-26T20:27:49.922231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931179+0000","last_change":"2024-04-26T20:27:49.931179+0000","last_active":"2024-04-26T20:27:49.931179+0000","last_peered":"2024-04-26T20:27:49.931179+0000","last_clean":"2024-04-26T20:27:49.931179+0000","last_became_active":"2024-04-26T20:27:49.930033+0000","last_became_peered":"2024-04-26T20:27:49.930033+0000","last_unstale":"2024-04-26T20:27:49.931179+0000","last_undegraded":"2024-04-26T20:27:49.931179+0000","last_fullsized":"2024-04-26T20:27:49.931179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926173+0000","last_change":"2024-04-26T20:27:49.926173+0000","last_active":"2024-04-26T20:27:49.926173+0000","last_peered":"2024-04-26T20:27:49.926173+0000","last_clean":"2024-04-26T20:27:49.926173+0000","last_became_active":"2024-04-26T20:27:49.925901+0000","last_became_peered":"2024-04-26T20:27:49.925901+0000","last_unstale":"2024-04-26T20:27:49.926173+0000","last_undegraded":"2024-04-26T20:27:49.926173+0000","last_fullsized":"2024-04-26T20:27:49.926173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932243+0000","last_change":"2024-04-26T20:27:49.932243+0000","last_active":"2024-04-26T20:27:49.932243+0000","last_peered":"2024-04-26T20:27:49.932243+0000","last_clean":"2024-04-26T20:27:49.932243+0000","last_became_active":"2024-04-26T20:27:49.931924+0000","last_became_peered":"2024-04-26T20:27:49.931924+0000","last_unstale":"2024-04-26T20:27:49.932243+0000","last_undegraded":"2024-04-26T20:27:49.932243+0000","last_fullsized":"2024-04-26T20:27:49.932243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928800+0000","last_change":"2024-04-26T20:27:49.924453+0000","last_active":"2024-04-26T20:27:49.928800+0000","last_peered":"2024-04-26T20:27:49.928800+0000","last_clean":"2024-04-26T20:27:49.928800+0000","last_became_active":"2024-04-26T20:27:49.924193+0000","last_became_peered":"2024-04-26T20:27:49.924193+0000","last_unstale":"2024-04-26T20:27:49.928800+0000","last_undegraded":"2024-04-26T20:27:49.928800+0000","last_fullsized":"2024-04-26T20:27:49.928800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922276+0000","last_change":"2024-04-26T20:27:49.917930+0000","last_active":"2024-04-26T20:27:49.922276+0000","last_peered":"2024-04-26T20:27:49.922276+0000","last_clean":"2024-04-26T20:27:49.922276+0000","last_became_active":"2024-04-26T20:27:49.914720+0000","last_became_peered":"2024-04-26T20:27:49.914720+0000","last_unstale":"2024-04-26T20:27:49.922276+0000","last_undegraded":"2024-04-26T20:27:49.922276+0000","last_fullsized":"2024-04-26T20:27:49.922276+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932579+0000","last_change":"2024-04-26T20:27:49.932579+0000","last_active":"2024-04-26T20:27:49.932579+0000","last_peered":"2024-04-26T20:27:49.932579+0000","last_clean":"2024-04-26T20:27:49.932579+0000","last_became_active":"2024-04-26T20:27:49.931863+0000","last_became_peered":"2024-04-26T20:27:49.931863+0000","last_unstale":"2024-04-26T20:27:49.932579+0000","last_undegraded":"2024-04-26T20:27:49.932579+0000","last_fullsized":"2024-04-26T20:27:49.932579+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.891489+0000","last_change":"2024-04-26T20:27:49.891489+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.891489+0000","last_undegraded":"2024-04-26T20:27:49.891489+0000","last_fullsized":"2024-04-26T20:27:49.891489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929160+0000","last_change":"2024-04-26T20:27:49.929160+0000","last_active":"2024-04-26T20:27:49.929160+0000","last_peered":"2024-04-26T20:27:49.929160+0000","last_clean":"2024-04-26T20:27:49.929160+0000","last_became_active":"2024-04-26T20:27:49.928881+0000","last_became_peered":"2024-04-26T20:27:49.928881+0000","last_unstale":"2024-04-26T20:27:49.929160+0000","last_undegraded":"2024-04-26T20:27:49.929160+0000","last_fullsized":"2024-04-26T20:27:49.929160+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913270+0000","last_change":"2024-04-26T20:27:49.913270+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913270+0000","last_undegraded":"2024-04-26T20:27:49.913270+0000","last_fullsized":"2024-04-26T20:27:49.913270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.921031+0000","last_change":"2024-04-26T20:27:49.921031+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.921031+0000","last_undegraded":"2024-04-26T20:27:49.921031+0000","last_fullsized":"2024-04-26T20:27:49.921031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927012+0000","last_change":"2024-04-26T20:27:49.927012+0000","last_active":"2024-04-26T20:27:49.927012+0000","last_peered":"2024-04-26T20:27:49.927012+0000","last_clean":"2024-04-26T20:27:49.927012+0000","last_became_active":"2024-04-26T20:27:49.926696+0000","last_became_peered":"2024-04-26T20:27:49.926696+0000","last_unstale":"2024-04-26T20:27:49.927012+0000","last_undegraded":"2024-04-26T20:27:49.927012+0000","last_fullsized":"2024-04-26T20:27:49.927012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911696+0000","last_change":"2024-04-26T20:27:49.902695+0000","last_active":"2024-04-26T20:27:49.911696+0000","last_peered":"2024-04-26T20:27:49.911696+0000","last_clean":"2024-04-26T20:27:49.911696+0000","last_became_active":"2024-04-26T20:27:49.901838+0000","last_became_peered":"2024-04-26T20:27:49.901838+0000","last_unstale":"2024-04-26T20:27:49.911696+0000","last_undegraded":"2024-04-26T20:27:49.911696+0000","last_fullsized":"2024-04-26T20:27:49.911696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:44.890223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926250+0000","last_change":"2024-04-26T20:27:49.926250+0000","last_active":"2024-04-26T20:27:49.926250+0000","last_peered":"2024-04-26T20:27:49.926250+0000","last_clean":"2024-04-26T20:27:49.926250+0000","last_became_active":"2024-04-26T20:27:49.925945+0000","last_became_peered":"2024-04-26T20:27:49.925945+0000","last_unstale":"2024-04-26T20:27:49.926250+0000","last_undegraded":"2024-04-26T20:27:49.926250+0000","last_fullsized":"2024-04-26T20:27:49.926250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928074+0000","last_change":"2024-04-26T20:27:49.928074+0000","last_active":"2024-04-26T20:27:49.928074+0000","last_peered":"2024-04-26T20:27:49.928074+0000","last_clean":"2024-04-26T20:27:49.928074+0000","last_became_active":"2024-04-26T20:27:49.920120+0000","last_became_peered":"2024-04-26T20:27:49.920120+0000","last_unstale":"2024-04-26T20:27:49.928074+0000","last_undegraded":"2024-04-26T20:27:49.928074+0000","last_fullsized":"2024-04-26T20:27:49.928074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916486+0000","last_change":"2024-04-26T20:27:49.906288+0000","last_active":"2024-04-26T20:27:49.916486+0000","last_peered":"2024-04-26T20:27:49.916486+0000","last_clean":"2024-04-26T20:27:49.916486+0000","last_became_active":"2024-04-26T20:27:49.905307+0000","last_became_peered":"2024-04-26T20:27:49.905307+0000","last_unstale":"2024-04-26T20:27:49.916486+0000","last_undegraded":"2024-04-26T20:27:49.916486+0000","last_fullsized":"2024-04-26T20:27:49.916486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913198+0000","last_change":"2024-04-26T20:27:49.913198+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913198+0000","last_undegraded":"2024-04-26T20:27:49.913198+0000","last_fullsized":"2024-04-26T20:27:49.913198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925927+0000","last_change":"2024-04-26T20:27:49.921300+0000","last_active":"2024-04-26T20:27:49.925927+0000","last_peered":"2024-04-26T20:27:49.925927+0000","last_clean":"2024-04-26T20:27:49.925927+0000","last_became_active":"2024-04-26T20:27:49.921025+0000","last_became_peered":"2024-04-26T20:27:49.921025+0000","last_unstale":"2024-04-26T20:27:49.925927+0000","last_undegraded":"2024-04-26T20:27:49.925927+0000","last_fullsized":"2024-04-26T20:27:49.925927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:04.556567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917029+0000","last_change":"2024-04-26T20:27:49.917029+0000","last_active":"2024-04-26T20:27:49.917029+0000","last_peered":"2024-04-26T20:27:49.917029+0000","last_clean":"2024-04-26T20:27:49.917029+0000","last_became_active":"2024-04-26T20:27:49.916684+0000","last_became_peered":"2024-04-26T20:27:49.916684+0000","last_unstale":"2024-04-26T20:27:49.917029+0000","last_undegraded":"2024-04-26T20:27:49.917029+0000","last_fullsized":"2024-04-26T20:27:49.917029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:34.091055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.893547+0000","last_change":"2024-04-26T20:27:49.893547+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.893547+0000","last_undegraded":"2024-04-26T20:27:49.893547+0000","last_fullsized":"2024-04-26T20:27:49.893547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.915570+0000","last_change":"2024-04-26T20:27:49.915570+0000","last_active":"2024-04-26T20:27:49.915570+0000","last_peered":"2024-04-26T20:27:49.915570+0000","last_clean":"2024-04-26T20:27:49.915570+0000","last_became_active":"2024-04-26T20:27:49.915277+0000","last_became_peered":"2024-04-26T20:27:49.915277+0000","last_unstale":"2024-04-26T20:27:49.915570+0000","last_undegraded":"2024-04-26T20:27:49.915570+0000","last_fullsized":"2024-04-26T20:27:49.915570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929083+0000","last_change":"2024-04-26T20:27:49.923260+0000","last_active":"2024-04-26T20:27:49.929083+0000","last_peered":"2024-04-26T20:27:49.929083+0000","last_clean":"2024-04-26T20:27:49.929083+0000","last_became_active":"2024-04-26T20:27:49.922900+0000","last_became_peered":"2024-04-26T20:27:49.922900+0000","last_unstale":"2024-04-26T20:27:49.929083+0000","last_undegraded":"2024-04-26T20:27:49.929083+0000","last_fullsized":"2024-04-26T20:27:49.929083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918298+0000","last_change":"2024-04-26T20:27:49.918298+0000","last_active":"2024-04-26T20:27:49.918298+0000","last_peered":"2024-04-26T20:27:49.918298+0000","last_clean":"2024-04-26T20:27:49.918298+0000","last_became_active":"2024-04-26T20:27:49.912487+0000","last_became_peered":"2024-04-26T20:27:49.912487+0000","last_unstale":"2024-04-26T20:27:49.918298+0000","last_undegraded":"2024-04-26T20:27:49.918298+0000","last_fullsized":"2024-04-26T20:27:49.918298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925689+0000","last_change":"2024-04-26T20:27:49.914198+0000","last_active":"2024-04-26T20:27:49.925689+0000","last_peered":"2024-04-26T20:27:49.925689+0000","last_clean":"2024-04-26T20:27:49.925689+0000","last_became_active":"2024-04-26T20:27:49.913905+0000","last_became_peered":"2024-04-26T20:27:49.913905+0000","last_unstale":"2024-04-26T20:27:49.925689+0000","last_undegraded":"2024-04-26T20:27:49.925689+0000","last_fullsized":"2024-04-26T20:27:49.925689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:34.250292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913586+0000","last_change":"2024-04-26T20:27:49.913586+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913586+0000","last_undegraded":"2024-04-26T20:27:49.913586+0000","last_fullsized":"2024-04-26T20:27:49.913586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.908366+0000","last_change":"2024-04-26T20:27:49.899934+0000","last_active":"2024-04-26T20:27:49.908366+0000","last_peered":"2024-04-26T20:27:49.908366+0000","last_clean":"2024-04-26T20:27:49.908366+0000","last_became_active":"2024-04-26T20:27:49.899653+0000","last_became_peered":"2024-04-26T20:27:49.899653+0000","last_unstale":"2024-04-26T20:27:49.908366+0000","last_undegraded":"2024-04-26T20:27:49.908366+0000","last_fullsized":"2024-04-26T20:27:49.908366+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.919146+0000","last_change":"2024-04-26T20:27:49.919146+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.919146+0000","last_undegraded":"2024-04-26T20:27:49.919146+0000","last_fullsized":"2024-04-26T20:27:49.919146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911517+0000","last_change":"2024-04-26T20:27:48.897659+0000","last_active":"2024-04-26T20:27:49.911517+0000","last_peered":"2024-04-26T20:27:49.911517+0000","last_clean":"2024-04-26T20:27:49.911517+0000","last_became_active":"2024-04-26T20:27:48.896450+0000","last_became_peered":"2024-04-26T20:27:48.896450+0000","last_unstale":"2024-04-26T20:27:49.911517+0000","last_undegraded":"2024-04-26T20:27:49.911517+0000","last_fullsized":"2024-04-26T20:27:49.911517+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892621+0000","last_change":"2024-04-26T20:27:49.892621+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892621+0000","last_undegraded":"2024-04-26T20:27:49.892621+0000","last_fullsized":"2024-04-26T20:27:49.892621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901581+0000","last_change":"2024-04-26T20:27:48.895441+0000","last_active":"2024-04-26T20:27:49.901581+0000","last_peered":"2024-04-26T20:27:49.901581+0000","last_clean":"2024-04-26T20:27:49.901581+0000","last_became_active":"2024-04-26T20:27:48.895118+0000","last_became_peered":"2024-04-26T20:27:48.895118+0000","last_unstale":"2024-04-26T20:27:49.901581+0000","last_undegraded":"2024-04-26T20:27:49.901581+0000","last_fullsized":"2024-04-26T20:27:49.901581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.890220+0000","last_change":"2024-04-26T20:27:49.890220+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.890220+0000","last_undegraded":"2024-04-26T20:27:49.890220+0000","last_fullsized":"2024-04-26T20:27:49.890220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895459+0000","last_change":"2024-04-26T20:27:48.894724+0000","last_active":"2024-04-26T20:27:49.895459+0000","last_peered":"2024-04-26T20:27:49.895459+0000","last_clean":"2024-04-26T20:27:49.895459+0000","last_became_active":"2024-04-26T20:27:48.893381+0000","last_became_peered":"2024-04-26T20:27:48.893381+0000","last_unstale":"2024-04-26T20:27:49.895459+0000","last_undegraded":"2024-04-26T20:27:49.895459+0000","last_fullsized":"2024-04-26T20:27:49.895459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.931161+0000","last_change":"2024-04-26T20:27:49.929224+0000","last_active":"2024-04-26T20:27:49.931161+0000","last_peered":"2024-04-26T20:27:49.931161+0000","last_clean":"2024-04-26T20:27:49.931161+0000","last_became_active":"2024-04-26T20:27:49.928934+0000","last_became_peered":"2024-04-26T20:27:49.928934+0000","last_unstale":"2024-04-26T20:27:49.931161+0000","last_undegraded":"2024-04-26T20:27:49.931161+0000","last_fullsized":"2024-04-26T20:27:49.931161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895394+0000","last_change":"2024-04-26T20:27:48.894846+0000","last_active":"2024-04-26T20:27:49.895394+0000","last_peered":"2024-04-26T20:27:49.895394+0000","last_clean":"2024-04-26T20:27:49.895394+0000","last_became_active":"2024-04-26T20:27:48.893698+0000","last_became_peered":"2024-04-26T20:27:48.893698+0000","last_unstale":"2024-04-26T20:27:49.895394+0000","last_undegraded":"2024-04-26T20:27:49.895394+0000","last_fullsized":"2024-04-26T20:27:49.895394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925175+0000","last_change":"2024-04-26T20:27:49.920847+0000","last_active":"2024-04-26T20:27:49.925175+0000","last_peered":"2024-04-26T20:27:49.925175+0000","last_clean":"2024-04-26T20:27:49.925175+0000","last_became_active":"2024-04-26T20:27:49.920522+0000","last_became_peered":"2024-04-26T20:27:49.920522+0000","last_unstale":"2024-04-26T20:27:49.925175+0000","last_undegraded":"2024-04-26T20:27:49.925175+0000","last_fullsized":"2024-04-26T20:27:49.925175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.930867+0000","last_change":"2024-04-26T20:27:48.891209+0000","last_active":"2024-04-26T20:27:49.930867+0000","last_peered":"2024-04-26T20:27:49.930867+0000","last_clean":"2024-04-26T20:27:49.930867+0000","last_became_active":"2024-04-26T20:27:48.890894+0000","last_became_peered":"2024-04-26T20:27:48.890894+0000","last_unstale":"2024-04-26T20:27:49.930867+0000","last_undegraded":"2024-04-26T20:27:49.930867+0000","last_fullsized":"2024-04-26T20:27:49.930867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928684+0000","last_change":"2024-04-26T20:27:49.920288+0000","last_active":"2024-04-26T20:27:49.928684+0000","last_peered":"2024-04-26T20:27:49.928684+0000","last_clean":"2024-04-26T20:27:49.928684+0000","last_became_active":"2024-04-26T20:27:49.919780+0000","last_became_peered":"2024-04-26T20:27:49.919780+0000","last_unstale":"2024-04-26T20:27:49.928684+0000","last_undegraded":"2024-04-26T20:27:49.928684+0000","last_fullsized":"2024-04-26T20:27:49.928684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922040+0000","last_change":"2024-04-26T20:27:48.897588+0000","last_active":"2024-04-26T20:27:49.922040+0000","last_peered":"2024-04-26T20:27:49.922040+0000","last_clean":"2024-04-26T20:27:49.922040+0000","last_became_active":"2024-04-26T20:27:48.896574+0000","last_became_peered":"2024-04-26T20:27:48.896574+0000","last_unstale":"2024-04-26T20:27:49.922040+0000","last_undegraded":"2024-04-26T20:27:49.922040+0000","last_fullsized":"2024-04-26T20:27:49.922040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:22.623531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.919624+0000","last_change":"2024-04-26T20:27:49.916775+0000","last_active":"2024-04-26T20:27:49.919624+0000","last_peered":"2024-04-26T20:27:49.919624+0000","last_clean":"2024-04-26T20:27:49.919624+0000","last_became_active":"2024-04-26T20:27:49.916312+0000","last_became_peered":"2024-04-26T20:27:49.916312+0000","last_unstale":"2024-04-26T20:27:49.919624+0000","last_undegraded":"2024-04-26T20:27:49.919624+0000","last_fullsized":"2024-04-26T20:27:49.919624+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:47.847414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895250+0000","last_change":"2024-04-26T20:27:48.898227+0000","last_active":"2024-04-26T20:27:49.895250+0000","last_peered":"2024-04-26T20:27:49.895250+0000","last_clean":"2024-04-26T20:27:49.895250+0000","last_became_active":"2024-04-26T20:27:48.897196+0000","last_became_peered":"2024-04-26T20:27:48.897196+0000","last_unstale":"2024-04-26T20:27:49.895250+0000","last_undegraded":"2024-04-26T20:27:49.895250+0000","last_fullsized":"2024-04-26T20:27:49.895250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:46.834398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.900144+0000","last_change":"2024-04-26T20:27:49.900144+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.900144+0000","last_undegraded":"2024-04-26T20:27:49.900144+0000","last_fullsized":"2024-04-26T20:27:49.900144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894039+0000","last_change":"2024-04-26T20:27:48.890830+0000","last_active":"2024-04-26T20:27:49.894039+0000","last_peered":"2024-04-26T20:27:49.894039+0000","last_clean":"2024-04-26T20:27:49.894039+0000","last_became_active":"2024-04-26T20:27:48.890544+0000","last_became_peered":"2024-04-26T20:27:48.890544+0000","last_unstale":"2024-04-26T20:27:49.894039+0000","last_undegraded":"2024-04-26T20:27:49.894039+0000","last_fullsized":"2024-04-26T20:27:49.894039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927244+0000","last_change":"2024-04-26T20:27:49.923399+0000","last_active":"2024-04-26T20:27:49.927244+0000","last_peered":"2024-04-26T20:27:49.927244+0000","last_clean":"2024-04-26T20:27:49.927244+0000","last_became_active":"2024-04-26T20:27:49.923127+0000","last_became_peered":"2024-04-26T20:27:49.923127+0000","last_unstale":"2024-04-26T20:27:49.927244+0000","last_undegraded":"2024-04-26T20:27:49.927244+0000","last_fullsized":"2024-04-26T20:27:49.927244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:24.111300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901840+0000","last_change":"2024-04-26T20:27:48.889003+0000","last_active":"2024-04-26T20:27:49.901840+0000","last_peered":"2024-04-26T20:27:49.901840+0000","last_clean":"2024-04-26T20:27:49.901840+0000","last_became_active":"2024-04-26T20:27:48.888708+0000","last_became_peered":"2024-04-26T20:27:48.888708+0000","last_unstale":"2024-04-26T20:27:49.901840+0000","last_undegraded":"2024-04-26T20:27:49.901840+0000","last_fullsized":"2024-04-26T20:27:49.901840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:17.802067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917978+0000","last_change":"2024-04-26T20:27:49.917978+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917978+0000","last_undegraded":"2024-04-26T20:27:49.917978+0000","last_fullsized":"2024-04-26T20:27:49.917978+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922887+0000","last_change":"2024-04-26T20:27:48.897655+0000","last_active":"2024-04-26T20:27:49.922887+0000","last_peered":"2024-04-26T20:27:49.922887+0000","last_clean":"2024-04-26T20:27:49.922887+0000","last_became_active":"2024-04-26T20:27:48.897232+0000","last_became_peered":"2024-04-26T20:27:48.897232+0000","last_unstale":"2024-04-26T20:27:49.922887+0000","last_undegraded":"2024-04-26T20:27:49.922887+0000","last_fullsized":"2024-04-26T20:27:49.922887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925441+0000","last_change":"2024-04-26T20:27:49.921057+0000","last_active":"2024-04-26T20:27:49.925441+0000","last_peered":"2024-04-26T20:27:49.925441+0000","last_clean":"2024-04-26T20:27:49.925441+0000","last_became_active":"2024-04-26T20:27:49.920747+0000","last_became_peered":"2024-04-26T20:27:49.920747+0000","last_unstale":"2024-04-26T20:27:49.925441+0000","last_undegraded":"2024-04-26T20:27:49.925441+0000","last_fullsized":"2024-04-26T20:27:49.925441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:40.387985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.922912+0000","last_change":"2024-04-26T20:27:48.895154+0000","last_active":"2024-04-26T20:27:49.922912+0000","last_peered":"2024-04-26T20:27:49.922912+0000","last_clean":"2024-04-26T20:27:49.922912+0000","last_became_active":"2024-04-26T20:27:48.894300+0000","last_became_peered":"2024-04-26T20:27:48.894300+0000","last_unstale":"2024-04-26T20:27:49.922912+0000","last_undegraded":"2024-04-26T20:27:49.922912+0000","last_fullsized":"2024-04-26T20:27:49.922912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.920045+0000","last_change":"2024-04-26T20:27:49.920045+0000","last_active":"2024-04-26T20:27:49.920045+0000","last_peered":"2024-04-26T20:27:49.920045+0000","last_clean":"2024-04-26T20:27:49.920045+0000","last_became_active":"2024-04-26T20:27:49.919739+0000","last_became_peered":"2024-04-26T20:27:49.919739+0000","last_unstale":"2024-04-26T20:27:49.920045+0000","last_undegraded":"2024-04-26T20:27:49.920045+0000","last_fullsized":"2024-04-26T20:27:49.920045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:08.674744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932303+0000","last_change":"2024-04-26T20:27:48.896675+0000","last_active":"2024-04-26T20:27:49.932303+0000","last_peered":"2024-04-26T20:27:49.932303+0000","last_clean":"2024-04-26T20:27:49.932303+0000","last_became_active":"2024-04-26T20:27:48.896362+0000","last_became_peered":"2024-04-26T20:27:48.896362+0000","last_unstale":"2024-04-26T20:27:49.932303+0000","last_undegraded":"2024-04-26T20:27:49.932303+0000","last_fullsized":"2024-04-26T20:27:49.932303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924165+0000","last_change":"2024-04-26T20:27:49.920198+0000","last_active":"2024-04-26T20:27:49.924165+0000","last_peered":"2024-04-26T20:27:49.924165+0000","last_clean":"2024-04-26T20:27:49.924165+0000","last_became_active":"2024-04-26T20:27:49.919936+0000","last_became_peered":"2024-04-26T20:27:49.919936+0000","last_unstale":"2024-04-26T20:27:49.924165+0000","last_undegraded":"2024-04-26T20:27:49.924165+0000","last_fullsized":"2024-04-26T20:27:49.924165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918703+0000","last_change":"2024-04-26T20:27:48.891091+0000","last_active":"2024-04-26T20:27:49.918703+0000","last_peered":"2024-04-26T20:27:49.918703+0000","last_clean":"2024-04-26T20:27:49.918703+0000","last_became_active":"2024-04-26T20:27:48.888052+0000","last_became_peered":"2024-04-26T20:27:48.888052+0000","last_unstale":"2024-04-26T20:27:49.918703+0000","last_undegraded":"2024-04-26T20:27:49.918703+0000","last_fullsized":"2024-04-26T20:27:49.918703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:35.082176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.923837+0000","last_change":"2024-04-26T20:27:49.917196+0000","last_active":"2024-04-26T20:27:49.923837+0000","last_peered":"2024-04-26T20:27:49.923837+0000","last_clean":"2024-04-26T20:27:49.923837+0000","last_became_active":"2024-04-26T20:27:49.916941+0000","last_became_peered":"2024-04-26T20:27:49.916941+0000","last_unstale":"2024-04-26T20:27:49.923837+0000","last_undegraded":"2024-04-26T20:27:49.923837+0000","last_fullsized":"2024-04-26T20:27:49.923837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895346+0000","last_change":"2024-04-26T20:27:48.894730+0000","last_active":"2024-04-26T20:27:49.895346+0000","last_peered":"2024-04-26T20:27:49.895346+0000","last_clean":"2024-04-26T20:27:49.895346+0000","last_became_active":"2024-04-26T20:27:48.894446+0000","last_became_peered":"2024-04-26T20:27:48.894446+0000","last_unstale":"2024-04-26T20:27:49.895346+0000","last_undegraded":"2024-04-26T20:27:49.895346+0000","last_fullsized":"2024-04-26T20:27:49.895346+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.913576+0000","last_change":"2024-04-26T20:27:49.913576+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.913576+0000","last_undegraded":"2024-04-26T20:27:49.913576+0000","last_fullsized":"2024-04-26T20:27:49.913576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904291+0000","last_change":"2024-04-26T20:27:48.886899+0000","last_active":"2024-04-26T20:27:49.904291+0000","last_peered":"2024-04-26T20:27:49.904291+0000","last_clean":"2024-04-26T20:27:49.904291+0000","last_became_active":"2024-04-26T20:27:48.886536+0000","last_became_peered":"2024-04-26T20:27:48.886536+0000","last_unstale":"2024-04-26T20:27:49.904291+0000","last_undegraded":"2024-04-26T20:27:49.904291+0000","last_fullsized":"2024-04-26T20:27:49.904291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929644+0000","last_change":"2024-04-26T20:27:49.918902+0000","last_active":"2024-04-26T20:27:49.929644+0000","last_peered":"2024-04-26T20:27:49.929644+0000","last_clean":"2024-04-26T20:27:49.929644+0000","last_became_active":"2024-04-26T20:27:49.918340+0000","last_became_peered":"2024-04-26T20:27:49.918340+0000","last_unstale":"2024-04-26T20:27:49.929644+0000","last_undegraded":"2024-04-26T20:27:49.929644+0000","last_fullsized":"2024-04-26T20:27:49.929644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917635+0000","last_change":"2024-04-26T20:27:48.886522+0000","last_active":"2024-04-26T20:27:49.917635+0000","last_peered":"2024-04-26T20:27:49.917635+0000","last_clean":"2024-04-26T20:27:49.917635+0000","last_became_active":"2024-04-26T20:27:48.885874+0000","last_became_peered":"2024-04-26T20:27:48.885874+0000","last_unstale":"2024-04-26T20:27:49.917635+0000","last_undegraded":"2024-04-26T20:27:49.917635+0000","last_fullsized":"2024-04-26T20:27:49.917635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.932240+0000","last_change":"2024-04-26T20:27:49.926821+0000","last_active":"2024-04-26T20:27:49.932240+0000","last_peered":"2024-04-26T20:27:49.932240+0000","last_clean":"2024-04-26T20:27:49.932240+0000","last_became_active":"2024-04-26T20:27:49.926564+0000","last_became_peered":"2024-04-26T20:27:49.926564+0000","last_unstale":"2024-04-26T20:27:49.932240+0000","last_undegraded":"2024-04-26T20:27:49.932240+0000","last_fullsized":"2024-04-26T20:27:49.932240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924746+0000","last_change":"2024-04-26T20:27:48.900526+0000","last_active":"2024-04-26T20:27:49.924746+0000","last_peered":"2024-04-26T20:27:49.924746+0000","last_clean":"2024-04-26T20:27:49.924746+0000","last_became_active":"2024-04-26T20:27:48.900110+0000","last_became_peered":"2024-04-26T20:27:48.900110+0000","last_unstale":"2024-04-26T20:27:49.924746+0000","last_undegraded":"2024-04-26T20:27:49.924746+0000","last_fullsized":"2024-04-26T20:27:49.924746+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:26.369631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.928774+0000","last_change":"2024-04-26T20:27:49.923959+0000","last_active":"2024-04-26T20:27:49.928774+0000","last_peered":"2024-04-26T20:27:49.928774+0000","last_clean":"2024-04-26T20:27:49.928774+0000","last_became_active":"2024-04-26T20:27:49.923064+0000","last_became_peered":"2024-04-26T20:27:49.923064+0000","last_unstale":"2024-04-26T20:27:49.928774+0000","last_undegraded":"2024-04-26T20:27:49.928774+0000","last_fullsized":"2024-04-26T20:27:49.928774+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895794+0000","last_change":"2024-04-26T20:27:48.899650+0000","last_active":"2024-04-26T20:27:49.895794+0000","last_peered":"2024-04-26T20:27:49.895794+0000","last_clean":"2024-04-26T20:27:49.895794+0000","last_became_active":"2024-04-26T20:27:48.899036+0000","last_became_peered":"2024-04-26T20:27:48.899036+0000","last_unstale":"2024-04-26T20:27:49.895794+0000","last_undegraded":"2024-04-26T20:27:49.895794+0000","last_fullsized":"2024-04-26T20:27:49.895794+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:33.237276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917237+0000","last_change":"2024-04-26T20:27:49.901218+0000","last_active":"2024-04-26T20:27:49.917237+0000","last_peered":"2024-04-26T20:27:49.917237+0000","last_clean":"2024-04-26T20:27:49.917237+0000","last_became_active":"2024-04-26T20:27:49.900732+0000","last_became_peered":"2024-04-26T20:27:49.900732+0000","last_unstale":"2024-04-26T20:27:49.917237+0000","last_undegraded":"2024-04-26T20:27:49.917237+0000","last_fullsized":"2024-04-26T20:27:49.917237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:25.433629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.926905+0000","last_change":"2024-04-26T20:27:48.883412+0000","last_active":"2024-04-26T20:27:49.926905+0000","last_peered":"2024-04-26T20:27:49.926905+0000","last_clean":"2024-04-26T20:27:49.926905+0000","last_became_active":"2024-04-26T20:27:48.882939+0000","last_became_peered":"2024-04-26T20:27:48.882939+0000","last_unstale":"2024-04-26T20:27:49.926905+0000","last_undegraded":"2024-04-26T20:27:49.926905+0000","last_fullsized":"2024-04-26T20:27:49.926905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.174009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.911670+0000","last_change":"2024-04-26T20:27:49.909503+0000","last_active":"2024-04-26T20:27:49.911670+0000","last_peered":"2024-04-26T20:27:49.911670+0000","last_clean":"2024-04-26T20:27:49.911670+0000","last_became_active":"2024-04-26T20:27:49.909122+0000","last_became_peered":"2024-04-26T20:27:49.909122+0000","last_unstale":"2024-04-26T20:27:49.911670+0000","last_undegraded":"2024-04-26T20:27:49.911670+0000","last_fullsized":"2024-04-26T20:27:49.911670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:46.024331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.895256+0000","last_change":"2024-04-26T20:27:48.885014+0000","last_active":"2024-04-26T20:27:49.895256+0000","last_peered":"2024-04-26T20:27:49.895256+0000","last_clean":"2024-04-26T20:27:49.895256+0000","last_became_active":"2024-04-26T20:27:48.884727+0000","last_became_peered":"2024-04-26T20:27:48.884727+0000","last_unstale":"2024-04-26T20:27:49.895256+0000","last_undegraded":"2024-04-26T20:27:49.895256+0000","last_fullsized":"2024-04-26T20:27:49.895256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917288+0000","last_change":"2024-04-26T20:27:49.917288+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917288+0000","last_undegraded":"2024-04-26T20:27:49.917288+0000","last_fullsized":"2024-04-26T20:27:49.917288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.916870+0000","last_change":"2024-04-26T20:27:48.880032+0000","last_active":"2024-04-26T20:27:49.916870+0000","last_peered":"2024-04-26T20:27:49.916870+0000","last_clean":"2024-04-26T20:27:49.916870+0000","last_became_active":"2024-04-26T20:27:48.878893+0000","last_became_peered":"2024-04-26T20:27:48.878893+0000","last_unstale":"2024-04-26T20:27:49.916870+0000","last_undegraded":"2024-04-26T20:27:49.916870+0000","last_fullsized":"2024-04-26T20:27:49.916870+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:04.775515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927225+0000","last_change":"2024-04-26T20:27:49.927225+0000","last_active":"2024-04-26T20:27:49.927225+0000","last_peered":"2024-04-26T20:27:49.927225+0000","last_clean":"2024-04-26T20:27:49.927225+0000","last_became_active":"2024-04-26T20:27:49.925074+0000","last_became_peered":"2024-04-26T20:27:49.925074+0000","last_unstale":"2024-04-26T20:27:49.927225+0000","last_undegraded":"2024-04-26T20:27:49.927225+0000","last_fullsized":"2024-04-26T20:27:49.927225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886368+0000","last_change":"2024-04-26T20:27:48.896812+0000","last_active":"2024-04-26T20:27:49.886368+0000","last_peered":"2024-04-26T20:27:49.886368+0000","last_clean":"2024-04-26T20:27:49.886368+0000","last_became_active":"2024-04-26T20:27:48.895863+0000","last_became_peered":"2024-04-26T20:27:48.895863+0000","last_unstale":"2024-04-26T20:27:49.886368+0000","last_undegraded":"2024-04-26T20:27:49.886368+0000","last_fullsized":"2024-04-26T20:27:49.886368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:43.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.927088+0000","last_change":"2024-04-26T20:27:49.927088+0000","last_active":"2024-04-26T20:27:49.927088+0000","last_peered":"2024-04-26T20:27:49.927088+0000","last_clean":"2024-04-26T20:27:49.927088+0000","last_became_active":"2024-04-26T20:27:49.926795+0000","last_became_peered":"2024-04-26T20:27:49.926795+0000","last_unstale":"2024-04-26T20:27:49.927088+0000","last_undegraded":"2024-04-26T20:27:49.927088+0000","last_fullsized":"2024-04-26T20:27:49.927088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.913876+0000","last_change":"2024-04-26T20:27:48.891475+0000","last_active":"2024-04-26T20:27:49.913876+0000","last_peered":"2024-04-26T20:27:49.913876+0000","last_clean":"2024-04-26T20:27:49.913876+0000","last_became_active":"2024-04-26T20:27:48.891058+0000","last_became_peered":"2024-04-26T20:27:48.891058+0000","last_unstale":"2024-04-26T20:27:49.913876+0000","last_undegraded":"2024-04-26T20:27:49.913876+0000","last_fullsized":"2024-04-26T20:27:49.913876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:23.923671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.929667+0000","last_change":"2024-04-26T20:27:49.921492+0000","last_active":"2024-04-26T20:27:49.929667+0000","last_peered":"2024-04-26T20:27:49.929667+0000","last_clean":"2024-04-26T20:27:49.929667+0000","last_became_active":"2024-04-26T20:27:49.921208+0000","last_became_peered":"2024-04-26T20:27:49.921208+0000","last_unstale":"2024-04-26T20:27:49.929667+0000","last_undegraded":"2024-04-26T20:27:49.929667+0000","last_fullsized":"2024-04-26T20:27:49.929667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.886522+0000","last_change":"2024-04-26T20:27:48.879281+0000","last_active":"2024-04-26T20:27:49.886522+0000","last_peered":"2024-04-26T20:27:49.886522+0000","last_clean":"2024-04-26T20:27:49.886522+0000","last_became_active":"2024-04-26T20:27:48.878964+0000","last_became_peered":"2024-04-26T20:27:48.878964+0000","last_unstale":"2024-04-26T20:27:49.886522+0000","last_undegraded":"2024-04-26T20:27:49.886522+0000","last_fullsized":"2024-04-26T20:27:49.886522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:32.854744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.892290+0000","last_change":"2024-04-26T20:27:49.892290+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.892290+0000","last_undegraded":"2024-04-26T20:27:49.892290+0000","last_fullsized":"2024-04-26T20:27:49.892290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924918+0000","last_change":"2024-04-26T20:27:48.896599+0000","last_active":"2024-04-26T20:27:49.924918+0000","last_peered":"2024-04-26T20:27:49.924918+0000","last_clean":"2024-04-26T20:27:49.924918+0000","last_became_active":"2024-04-26T20:27:48.896224+0000","last_became_peered":"2024-04-26T20:27:48.896224+0000","last_unstale":"2024-04-26T20:27:49.924918+0000","last_undegraded":"2024-04-26T20:27:49.924918+0000","last_fullsized":"2024-04-26T20:27:49.924918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:52.021385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.918360+0000","last_change":"2024-04-26T20:27:49.918360+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.918360+0000","last_undegraded":"2024-04-26T20:27:49.918360+0000","last_fullsized":"2024-04-26T20:27:49.918360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.887587+0000","last_change":"2024-04-26T20:27:48.892674+0000","last_active":"2024-04-26T20:27:49.887587+0000","last_peered":"2024-04-26T20:27:49.887587+0000","last_clean":"2024-04-26T20:27:49.887587+0000","last_became_active":"2024-04-26T20:27:48.892364+0000","last_became_peered":"2024-04-26T20:27:48.892364+0000","last_unstale":"2024-04-26T20:27:49.887587+0000","last_undegraded":"2024-04-26T20:27:49.887587+0000","last_fullsized":"2024-04-26T20:27:49.887587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:56.403870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925751+0000","last_change":"2024-04-26T20:27:49.918395+0000","last_active":"2024-04-26T20:27:49.925751+0000","last_peered":"2024-04-26T20:27:49.925751+0000","last_clean":"2024-04-26T20:27:49.925751+0000","last_became_active":"2024-04-26T20:27:49.918111+0000","last_became_peered":"2024-04-26T20:27:49.918111+0000","last_unstale":"2024-04-26T20:27:49.925751+0000","last_undegraded":"2024-04-26T20:27:49.925751+0000","last_fullsized":"2024-04-26T20:27:49.925751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.903165+0000","last_change":"2024-04-26T20:27:48.886260+0000","last_active":"2024-04-26T20:27:49.903165+0000","last_peered":"2024-04-26T20:27:49.903165+0000","last_clean":"2024-04-26T20:27:49.903165+0000","last_became_active":"2024-04-26T20:27:48.885935+0000","last_became_peered":"2024-04-26T20:27:48.885935+0000","last_unstale":"2024-04-26T20:27:49.903165+0000","last_undegraded":"2024-04-26T20:27:49.903165+0000","last_fullsized":"2024-04-26T20:27:49.903165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.924077+0000","last_change":"2024-04-26T20:27:49.919086+0000","last_active":"2024-04-26T20:27:49.924077+0000","last_peered":"2024-04-26T20:27:49.924077+0000","last_clean":"2024-04-26T20:27:49.924077+0000","last_became_active":"2024-04-26T20:27:49.913716+0000","last_became_peered":"2024-04-26T20:27:49.913716+0000","last_unstale":"2024-04-26T20:27:49.924077+0000","last_undegraded":"2024-04-26T20:27:49.924077+0000","last_fullsized":"2024-04-26T20:27:49.924077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.901017+0000","last_change":"2024-04-26T20:27:48.892408+0000","last_active":"2024-04-26T20:27:49.901017+0000","last_peered":"2024-04-26T20:27:49.901017+0000","last_clean":"2024-04-26T20:27:49.901017+0000","last_became_active":"2024-04-26T20:27:48.891998+0000","last_became_peered":"2024-04-26T20:27:48.891998+0000","last_unstale":"2024-04-26T20:27:49.901017+0000","last_undegraded":"2024-04-26T20:27:49.901017+0000","last_fullsized":"2024-04-26T20:27:49.901017+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.912958+0000","last_change":"2024-04-26T20:27:49.912958+0000","last_active":"2024-04-26T20:27:49.912958+0000","last_peered":"2024-04-26T20:27:49.912958+0000","last_clean":"2024-04-26T20:27:49.912958+0000","last_became_active":"2024-04-26T20:27:49.912656+0000","last_became_peered":"2024-04-26T20:27:49.912656+0000","last_unstale":"2024-04-26T20:27:49.912958+0000","last_undegraded":"2024-04-26T20:27:49.912958+0000","last_fullsized":"2024-04-26T20:27:49.912958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.904813+0000","last_change":"2024-04-26T20:27:48.883806+0000","last_active":"2024-04-26T20:27:49.904813+0000","last_peered":"2024-04-26T20:27:49.904813+0000","last_clean":"2024-04-26T20:27:49.904813+0000","last_became_active":"2024-04-26T20:27:48.883495+0000","last_became_peered":"2024-04-26T20:27:48.883495+0000","last_unstale":"2024-04-26T20:27:49.904813+0000","last_undegraded":"2024-04-26T20:27:49.904813+0000","last_fullsized":"2024-04-26T20:27:49.904813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918725+0000","last_change":"2024-04-26T20:27:49.907230+0000","last_active":"2024-04-26T20:27:49.918725+0000","last_peered":"2024-04-26T20:27:49.918725+0000","last_clean":"2024-04-26T20:27:49.918725+0000","last_became_active":"2024-04-26T20:27:49.905877+0000","last_became_peered":"2024-04-26T20:27:49.905877+0000","last_unstale":"2024-04-26T20:27:49.918725+0000","last_undegraded":"2024-04-26T20:27:49.918725+0000","last_fullsized":"2024-04-26T20:27:49.918725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:58:38.549522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.894422+0000","last_change":"2024-04-26T20:27:48.893801+0000","last_active":"2024-04-26T20:27:49.894422+0000","last_peered":"2024-04-26T20:27:49.894422+0000","last_clean":"2024-04-26T20:27:49.894422+0000","last_became_active":"2024-04-26T20:27:48.893424+0000","last_became_peered":"2024-04-26T20:27:48.893424+0000","last_unstale":"2024-04-26T20:27:49.894422+0000","last_undegraded":"2024-04-26T20:27:49.894422+0000","last_fullsized":"2024-04-26T20:27:49.894422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-26T20:27:49.917924+0000","last_change":"2024-04-26T20:27:49.917924+0000","last_active":"2024-04-26T20:27:48.849301+0000","last_peered":"2024-04-26T20:27:48.849301+0000","last_clean":"2024-04-26T20:27:48.849301+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:27:49.917924+0000","last_undegraded":"2024-04-26T20:27:49.917924+0000","last_fullsized":"2024-04-26T20:27:49.917924+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.898659+0000","last_change":"2024-04-26T20:27:48.885205+0000","last_active":"2024-04-26T20:27:49.898659+0000","last_peered":"2024-04-26T20:27:49.898659+0000","last_clean":"2024-04-26T20:27:49.898659+0000","last_became_active":"2024-04-26T20:27:48.884908+0000","last_became_peered":"2024-04-26T20:27:48.884908+0000","last_unstale":"2024-04-26T20:27:49.898659+0000","last_undegraded":"2024-04-26T20:27:49.898659+0000","last_fullsized":"2024-04-26T20:27:49.898659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.925449+0000","last_change":"2024-04-26T20:27:49.925449+0000","last_active":"2024-04-26T20:27:49.925449+0000","last_peered":"2024-04-26T20:27:49.925449+0000","last_clean":"2024-04-26T20:27:49.925449+0000","last_became_active":"2024-04-26T20:27:49.925098+0000","last_became_peered":"2024-04-26T20:27:49.925098+0000","last_unstale":"2024-04-26T20:27:49.925449+0000","last_undegraded":"2024-04-26T20:27:49.925449+0000","last_fullsized":"2024-04-26T20:27:49.925449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.918901+0000","last_change":"2024-04-26T20:27:48.889736+0000","last_active":"2024-04-26T20:27:49.918901+0000","last_peered":"2024-04-26T20:27:49.918901+0000","last_clean":"2024-04-26T20:27:49.918901+0000","last_became_active":"2024-04-26T20:27:48.889103+0000","last_became_peered":"2024-04-26T20:27:48.889103+0000","last_unstale":"2024-04-26T20:27:49.918901+0000","last_undegraded":"2024-04-26T20:27:49.918901+0000","last_fullsized":"2024-04-26T20:27:49.918901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:27:49.917420+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:27:49.917420+0000","last_peered":"2024-04-26T20:27:49.917420+0000","last_clean":"2024-04-26T20:27:49.917420+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:27:49.917420+0000","last_undegraded":"2024-04-26T20:27:49.917420+0000","last_fullsized":"2024-04-26T20:27:49.917420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1470,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":176371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":766739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:27:57.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:57.511 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:27:59.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:27:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.727+0000 7fd93a600640 1 Processor -- start 2024-04-26T20:27:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.727+0000 7fd93a600640 1 -- start start 2024-04-26T20:27:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.728+0000 7fd93a600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 0x7fd9340d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd93a600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd93415f0d0 0x7fd9340cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd93a600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9340d3a10 con 0x7fd9340b7580 2024-04-26T20:27:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd93a600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd9340c8830 con 0x7fd93415f0d0 2024-04-26T20:27:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd93a600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd9340d3cb0 con 0x7fd9340b71a0 2024-04-26T20:27:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd93415f0d0 0x7fd9340cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd93415f0d0 0x7fd9340cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2024-04-26T20:27:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 -- 172.21.15.43:0/994810062 learned_addr learned my addr 172.21.15.43:0/994810062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933400640 1 --2- 172.21.15.43:0/994810062 >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 0x7fd9340d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 -- 172.21.15.43:0/994810062 >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 msgr2=0x7fd9340d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 --2- 172.21.15.43:0/994810062 >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 0x7fd9340d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 -- 172.21.15.43:0/994810062 >> v1:172.21.15.43:6789/0 conn(0x7fd9340b7580 legacy=0x7fd93415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933e00640 1 -- 172.21.15.43:0/994810062 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9340c8af0 con 0x7fd93415f0d0 2024-04-26T20:27:59.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.729+0000 7fd933400640 1 --2- 172.21.15.43:0/994810062 >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 0x7fd9340d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd933e00640 1 --2- 172.21.15.43:0/994810062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93415f0d0 0x7fd9340cf930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd93408d350 tx=0x7fd92c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703ef690aa98db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd932000640 1 -- 172.21.15.43:0/994810062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c037070 con 0x7fd93415f0d0 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd932000640 1 -- 172.21.15.43:0/994810062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c032440 con 0x7fd93415f0d0 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd932000640 1 -- 172.21.15.43:0/994810062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c032810 con 0x7fd93415f0d0 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd93a600640 1 -- 172.21.15.43:0/994810062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93415f0d0 msgr2=0x7fd9340cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.731+0000 7fd93a600640 1 --2- 172.21.15.43:0/994810062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93415f0d0 0x7fd9340cf930 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd93408d350 tx=0x7fd92c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 -- 172.21.15.43:0/994810062 shutdown_connections 2024-04-26T20:27:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 --2- 172.21.15.43:0/994810062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93415f0d0 0x7fd9340cf930 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 --2- 172.21.15.43:0/994810062 >> v2:172.21.15.164:3300/0 conn(0x7fd9340b71a0 0x7fd9340d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 -- 172.21.15.43:0/994810062 >> 172.21.15.43:0/994810062 conn(0x7fd9340d4d70 msgr2=0x7fd9340d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 -- 172.21.15.43:0/994810062 shutdown_connections 2024-04-26T20:27:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.732+0000 7fd93a600640 1 -- 172.21.15.43:0/994810062 wait complete. 2024-04-26T20:27:59.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.733+0000 7fd93a600640 1 Processor -- start 2024-04-26T20:27:59.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 -- start start 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 0x7fd9340a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd93415f0d0 0x7fd93409fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd93415f500 con 0x7fd9340b7580 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340c90b0 con 0x7fd9340b71a0 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.734+0000 7fd93a600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd93415f680 con 0x7fd93415f0d0 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:59.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36666/0 (socket says 172.21.15.43:36666) 2024-04-26T20:27:59.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 -- 172.21.15.43:0/3430608983 learned_addr learned my addr 172.21.15.43:0/3430608983 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 -- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd93415f0d0 msgr2=0x7fd93409fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd93415f0d0 0x7fd93409fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd933400640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 0x7fd9340a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 -- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 msgr2=0x7fd9340a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 0x7fd9340a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd932a00640 1 -- 172.21.15.43:0/3430608983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9340a1f10 con 0x7fd9340b7580 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.735+0000 7fd933400640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 0x7fd9340a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:27:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd932a00640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c760 tx=0x7fd91c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c016020 con 0x7fd9340b7580 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91c006d50 con 0x7fd9340b7580 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c003bb0 con 0x7fd9340b7580 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9340a21f0 con 0x7fd9340b7580 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.736+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9340a27b0 con 0x7fd9340b7580 2024-04-26T20:27:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.737+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd934069350 con 0x7fd9340b7580 2024-04-26T20:27:59.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.738+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd91c00a850 con 0x7fd9340b7580 2024-04-26T20:27:59.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.739+0000 7fd92b400640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 0x7fd91006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:27:59.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.740+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd91c08ba10 con 0x7fd9340b7580 2024-04-26T20:27:59.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.740+0000 7fd933400640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 0x7fd91006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:27:59.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.740+0000 7fd933400640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 0x7fd91006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:27:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.744+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd91c020020 con 0x7fd9340b7580 2024-04-26T20:27:59.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.950+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd93408ee00 con 0x7fd91006c8f0 2024-04-26T20:27:59.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.970+0000 7fd92b400640 1 -- 172.21.15.43:0/3430608983 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fd93408ee00 con 0x7fd91006c8f0 2024-04-26T20:27:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:27:59.976 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 msgr2=0x7fd91006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 0x7fd91006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924008040 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 msgr2=0x7fd9341513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c760 tx=0x7fd91c00cc20 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 shutdown_connections 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd91006c8f0 0x7fd91006edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd93415f0d0 0x7fd93409fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9340b7580 0x7fd9341513f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 --2- 172.21.15.43:0/3430608983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9340b71a0 0x7fd9340a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:27:59.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 >> 172.21.15.43:0/3430608983 conn(0x7fd9340d4d70 msgr2=0x7fd93406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:27:59.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.976+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 shutdown_connections 2024-04-26T20:27:59.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:27:59.977+0000 7fd93a600640 1 -- 172.21.15.43:0/3430608983 wait complete. 2024-04-26T20:27:59.999 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-04-26T20:27:58.900655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237488,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737232,"statfs":{"total":773094113280,"available":772850925568,"internally_reserved":0,"allocated":4759552,"data_stored":2707396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.016794"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091878+0000","last_change":"2024-04-26T20:27:49.918830+0000","last_active":"2024-04-26T20:27:51.091878+0000","last_peered":"2024-04-26T20:27:51.091878+0000","last_clean":"2024-04-26T20:27:51.091878+0000","last_became_active":"2024-04-26T20:27:49.918530+0000","last_became_peered":"2024-04-26T20:27:49.918530+0000","last_unstale":"2024-04-26T20:27:51.091878+0000","last_undegraded":"2024-04-26T20:27:51.091878+0000","last_fullsized":"2024-04-26T20:27:51.091878+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:27.969586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091347+0000","last_change":"2024-04-26T20:27:49.911220+0000","last_active":"2024-04-26T20:27:51.091347+0000","last_peered":"2024-04-26T20:27:51.091347+0000","last_clean":"2024-04-26T20:27:51.091347+0000","last_became_active":"2024-04-26T20:27:49.910949+0000","last_became_peered":"2024-04-26T20:27:49.910949+0000","last_unstale":"2024-04-26T20:27:51.091347+0000","last_undegraded":"2024-04-26T20:27:51.091347+0000","last_fullsized":"2024-04-26T20:27:51.091347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:32.634445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088158+0000","last_change":"2024-04-26T20:27:49.926512+0000","last_active":"2024-04-26T20:27:51.088158+0000","last_peered":"2024-04-26T20:27:51.088158+0000","last_clean":"2024-04-26T20:27:51.088158+0000","last_became_active":"2024-04-26T20:27:49.925884+0000","last_became_peered":"2024-04-26T20:27:49.925884+0000","last_unstale":"2024-04-26T20:27:51.088158+0000","last_undegraded":"2024-04-26T20:27:51.088158+0000","last_fullsized":"2024-04-26T20:27:51.088158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913253+0000","last_change":"2024-04-26T20:27:49.923178+0000","last_active":"2024-04-26T20:27:50.913253+0000","last_peered":"2024-04-26T20:27:50.913253+0000","last_clean":"2024-04-26T20:27:50.913253+0000","last_became_active":"2024-04-26T20:27:49.920812+0000","last_became_peered":"2024-04-26T20:27:49.920812+0000","last_unstale":"2024-04-26T20:27:50.913253+0000","last_undegraded":"2024-04-26T20:27:50.913253+0000","last_fullsized":"2024-04-26T20:27:50.913253+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:37.412959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913026+0000","last_change":"2024-04-26T20:27:49.922091+0000","last_active":"2024-04-26T20:27:50.913026+0000","last_peered":"2024-04-26T20:27:50.913026+0000","last_clean":"2024-04-26T20:27:50.913026+0000","last_became_active":"2024-04-26T20:27:49.921810+0000","last_became_peered":"2024-04-26T20:27:49.921810+0000","last_unstale":"2024-04-26T20:27:50.913026+0000","last_undegraded":"2024-04-26T20:27:50.913026+0000","last_fullsized":"2024-04-26T20:27:50.913026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:52.454765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.087661+0000","last_change":"2024-04-26T20:27:49.902670+0000","last_active":"2024-04-26T20:27:51.087661+0000","last_peered":"2024-04-26T20:27:51.087661+0000","last_clean":"2024-04-26T20:27:51.087661+0000","last_became_active":"2024-04-26T20:27:49.902336+0000","last_became_peered":"2024-04-26T20:27:49.902336+0000","last_unstale":"2024-04-26T20:27:51.087661+0000","last_undegraded":"2024-04-26T20:27:51.087661+0000","last_fullsized":"2024-04-26T20:27:51.087661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088698+0000","last_change":"2024-04-26T20:27:49.922231+0000","last_active":"2024-04-26T20:27:51.088698+0000","last_peered":"2024-04-26T20:27:51.088698+0000","last_clean":"2024-04-26T20:27:51.088698+0000","last_became_active":"2024-04-26T20:27:49.921927+0000","last_became_peered":"2024-04-26T20:27:49.921927+0000","last_unstale":"2024-04-26T20:27:51.088698+0000","last_undegraded":"2024-04-26T20:27:51.088698+0000","last_fullsized":"2024-04-26T20:27:51.088698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:47.609089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090642+0000","last_change":"2024-04-26T20:27:49.931179+0000","last_active":"2024-04-26T20:27:51.090642+0000","last_peered":"2024-04-26T20:27:51.090642+0000","last_clean":"2024-04-26T20:27:51.090642+0000","last_became_active":"2024-04-26T20:27:49.930033+0000","last_became_peered":"2024-04-26T20:27:49.930033+0000","last_unstale":"2024-04-26T20:27:51.090642+0000","last_undegraded":"2024-04-26T20:27:51.090642+0000","last_fullsized":"2024-04-26T20:27:51.090642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:52.868375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089238+0000","last_change":"2024-04-26T20:27:49.926173+0000","last_active":"2024-04-26T20:27:51.089238+0000","last_peered":"2024-04-26T20:27:51.089238+0000","last_clean":"2024-04-26T20:27:51.089238+0000","last_became_active":"2024-04-26T20:27:49.925901+0000","last_became_peered":"2024-04-26T20:27:49.925901+0000","last_unstale":"2024-04-26T20:27:51.089238+0000","last_undegraded":"2024-04-26T20:27:51.089238+0000","last_fullsized":"2024-04-26T20:27:51.089238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:18.652471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090196+0000","last_change":"2024-04-26T20:27:49.932243+0000","last_active":"2024-04-26T20:27:51.090196+0000","last_peered":"2024-04-26T20:27:51.090196+0000","last_clean":"2024-04-26T20:27:51.090196+0000","last_became_active":"2024-04-26T20:27:49.931924+0000","last_became_peered":"2024-04-26T20:27:49.931924+0000","last_unstale":"2024-04-26T20:27:51.090196+0000","last_undegraded":"2024-04-26T20:27:51.090196+0000","last_fullsized":"2024-04-26T20:27:51.090196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:17.876179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089478+0000","last_change":"2024-04-26T20:27:49.924453+0000","last_active":"2024-04-26T20:27:51.089478+0000","last_peered":"2024-04-26T20:27:51.089478+0000","last_clean":"2024-04-26T20:27:51.089478+0000","last_became_active":"2024-04-26T20:27:49.924193+0000","last_became_peered":"2024-04-26T20:27:49.924193+0000","last_unstale":"2024-04-26T20:27:51.089478+0000","last_undegraded":"2024-04-26T20:27:51.089478+0000","last_fullsized":"2024-04-26T20:27:51.089478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:32.634445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.087958+0000","last_change":"2024-04-26T20:27:49.917930+0000","last_active":"2024-04-26T20:27:51.087958+0000","last_peered":"2024-04-26T20:27:51.087958+0000","last_clean":"2024-04-26T20:27:51.087958+0000","last_became_active":"2024-04-26T20:27:49.914720+0000","last_became_peered":"2024-04-26T20:27:49.914720+0000","last_unstale":"2024-04-26T20:27:51.087958+0000","last_undegraded":"2024-04-26T20:27:51.087958+0000","last_fullsized":"2024-04-26T20:27:51.087958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090031+0000","last_change":"2024-04-26T20:27:49.932579+0000","last_active":"2024-04-26T20:27:51.090031+0000","last_peered":"2024-04-26T20:27:51.090031+0000","last_clean":"2024-04-26T20:27:51.090031+0000","last_became_active":"2024-04-26T20:27:49.931863+0000","last_became_peered":"2024-04-26T20:27:49.931863+0000","last_unstale":"2024-04-26T20:27:51.090031+0000","last_undegraded":"2024-04-26T20:27:51.090031+0000","last_fullsized":"2024-04-26T20:27:51.090031+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:55.419963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090733+0000","last_change":"2024-04-26T20:27:49.927774+0000","last_active":"2024-04-26T20:27:51.090733+0000","last_peered":"2024-04-26T20:27:51.090733+0000","last_clean":"2024-04-26T20:27:51.090733+0000","last_became_active":"2024-04-26T20:27:49.927508+0000","last_became_peered":"2024-04-26T20:27:49.927508+0000","last_unstale":"2024-04-26T20:27:51.090733+0000","last_undegraded":"2024-04-26T20:27:51.090733+0000","last_fullsized":"2024-04-26T20:27:51.090733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088968+0000","last_change":"2024-04-26T20:27:49.929160+0000","last_active":"2024-04-26T20:27:51.088968+0000","last_peered":"2024-04-26T20:27:51.088968+0000","last_clean":"2024-04-26T20:27:51.088968+0000","last_became_active":"2024-04-26T20:27:49.928881+0000","last_became_peered":"2024-04-26T20:27:49.928881+0000","last_unstale":"2024-04-26T20:27:51.088968+0000","last_undegraded":"2024-04-26T20:27:51.088968+0000","last_fullsized":"2024-04-26T20:27:51.088968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.625061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.917430+0000","last_change":"2024-04-26T20:27:49.933981+0000","last_active":"2024-04-26T20:27:50.917430+0000","last_peered":"2024-04-26T20:27:50.917430+0000","last_clean":"2024-04-26T20:27:50.917430+0000","last_became_active":"2024-04-26T20:27:49.933611+0000","last_became_peered":"2024-04-26T20:27:49.933611+0000","last_unstale":"2024-04-26T20:27:50.917430+0000","last_undegraded":"2024-04-26T20:27:50.917430+0000","last_fullsized":"2024-04-26T20:27:50.917430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.920635+0000","last_change":"2024-04-26T20:27:49.942644+0000","last_active":"2024-04-26T20:27:50.920635+0000","last_peered":"2024-04-26T20:27:50.920635+0000","last_clean":"2024-04-26T20:27:50.920635+0000","last_became_active":"2024-04-26T20:27:49.942262+0000","last_became_peered":"2024-04-26T20:27:49.942262+0000","last_unstale":"2024-04-26T20:27:50.920635+0000","last_undegraded":"2024-04-26T20:27:50.920635+0000","last_fullsized":"2024-04-26T20:27:50.920635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:32.634445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919154+0000","last_change":"2024-04-26T20:27:49.927012+0000","last_active":"2024-04-26T20:27:50.919154+0000","last_peered":"2024-04-26T20:27:50.919154+0000","last_clean":"2024-04-26T20:27:50.919154+0000","last_became_active":"2024-04-26T20:27:49.926696+0000","last_became_peered":"2024-04-26T20:27:49.926696+0000","last_unstale":"2024-04-26T20:27:50.919154+0000","last_undegraded":"2024-04-26T20:27:50.919154+0000","last_fullsized":"2024-04-26T20:27:50.919154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913249+0000","last_change":"2024-04-26T20:27:49.902695+0000","last_active":"2024-04-26T20:27:50.913249+0000","last_peered":"2024-04-26T20:27:50.913249+0000","last_clean":"2024-04-26T20:27:50.913249+0000","last_became_active":"2024-04-26T20:27:49.901838+0000","last_became_peered":"2024-04-26T20:27:49.901838+0000","last_unstale":"2024-04-26T20:27:50.913249+0000","last_undegraded":"2024-04-26T20:27:50.913249+0000","last_fullsized":"2024-04-26T20:27:50.913249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:02.574576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.917437+0000","last_change":"2024-04-26T20:27:49.926250+0000","last_active":"2024-04-26T20:27:50.917437+0000","last_peered":"2024-04-26T20:27:50.917437+0000","last_clean":"2024-04-26T20:27:50.917437+0000","last_became_active":"2024-04-26T20:27:49.925945+0000","last_became_peered":"2024-04-26T20:27:49.925945+0000","last_unstale":"2024-04-26T20:27:50.917437+0000","last_undegraded":"2024-04-26T20:27:50.917437+0000","last_fullsized":"2024-04-26T20:27:50.917437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:58.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088647+0000","last_change":"2024-04-26T20:27:49.928074+0000","last_active":"2024-04-26T20:27:51.088647+0000","last_peered":"2024-04-26T20:27:51.088647+0000","last_clean":"2024-04-26T20:27:51.088647+0000","last_became_active":"2024-04-26T20:27:49.920120+0000","last_became_peered":"2024-04-26T20:27:49.920120+0000","last_unstale":"2024-04-26T20:27:51.088647+0000","last_undegraded":"2024-04-26T20:27:51.088647+0000","last_fullsized":"2024-04-26T20:27:51.088647+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:42.061423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091629+0000","last_change":"2024-04-26T20:27:49.906288+0000","last_active":"2024-04-26T20:27:51.091629+0000","last_peered":"2024-04-26T20:27:51.091629+0000","last_clean":"2024-04-26T20:27:51.091629+0000","last_became_active":"2024-04-26T20:27:49.905307+0000","last_became_peered":"2024-04-26T20:27:49.905307+0000","last_unstale":"2024-04-26T20:27:51.091629+0000","last_undegraded":"2024-04-26T20:27:51.091629+0000","last_fullsized":"2024-04-26T20:27:51.091629+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:40.387985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918734+0000","last_change":"2024-04-26T20:27:49.930083+0000","last_active":"2024-04-26T20:27:50.918734+0000","last_peered":"2024-04-26T20:27:50.918734+0000","last_clean":"2024-04-26T20:27:50.918734+0000","last_became_active":"2024-04-26T20:27:49.926502+0000","last_became_peered":"2024-04-26T20:27:49.926502+0000","last_unstale":"2024-04-26T20:27:50.918734+0000","last_undegraded":"2024-04-26T20:27:50.918734+0000","last_fullsized":"2024-04-26T20:27:50.918734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:42.061423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.912831+0000","last_change":"2024-04-26T20:27:49.921300+0000","last_active":"2024-04-26T20:27:50.912831+0000","last_peered":"2024-04-26T20:27:50.912831+0000","last_clean":"2024-04-26T20:27:50.912831+0000","last_became_active":"2024-04-26T20:27:49.921025+0000","last_became_peered":"2024-04-26T20:27:49.921025+0000","last_unstale":"2024-04-26T20:27:50.912831+0000","last_undegraded":"2024-04-26T20:27:50.912831+0000","last_fullsized":"2024-04-26T20:27:50.912831+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:11.757585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918821+0000","last_change":"2024-04-26T20:27:49.917029+0000","last_active":"2024-04-26T20:27:50.918821+0000","last_peered":"2024-04-26T20:27:50.918821+0000","last_clean":"2024-04-26T20:27:50.918821+0000","last_became_active":"2024-04-26T20:27:49.916684+0000","last_became_peered":"2024-04-26T20:27:49.916684+0000","last_unstale":"2024-04-26T20:27:50.918821+0000","last_undegraded":"2024-04-26T20:27:50.918821+0000","last_fullsized":"2024-04-26T20:27:50.918821+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:24.936687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088387+0000","last_change":"2024-04-26T20:27:49.925661+0000","last_active":"2024-04-26T20:27:51.088387+0000","last_peered":"2024-04-26T20:27:51.088387+0000","last_clean":"2024-04-26T20:27:51.088387+0000","last_became_active":"2024-04-26T20:27:49.925332+0000","last_became_peered":"2024-04-26T20:27:49.925332+0000","last_unstale":"2024-04-26T20:27:51.088387+0000","last_undegraded":"2024-04-26T20:27:51.088387+0000","last_fullsized":"2024-04-26T20:27:51.088387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:53.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918555+0000","last_change":"2024-04-26T20:27:49.915570+0000","last_active":"2024-04-26T20:27:50.918555+0000","last_peered":"2024-04-26T20:27:50.918555+0000","last_clean":"2024-04-26T20:27:50.918555+0000","last_became_active":"2024-04-26T20:27:49.915277+0000","last_became_peered":"2024-04-26T20:27:49.915277+0000","last_unstale":"2024-04-26T20:27:50.918555+0000","last_undegraded":"2024-04-26T20:27:50.918555+0000","last_fullsized":"2024-04-26T20:27:50.918555+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:25.233401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089370+0000","last_change":"2024-04-26T20:27:49.923260+0000","last_active":"2024-04-26T20:27:51.089370+0000","last_peered":"2024-04-26T20:27:51.089370+0000","last_clean":"2024-04-26T20:27:51.089370+0000","last_became_active":"2024-04-26T20:27:49.922900+0000","last_became_peered":"2024-04-26T20:27:49.922900+0000","last_unstale":"2024-04-26T20:27:51.089370+0000","last_undegraded":"2024-04-26T20:27:51.089370+0000","last_fullsized":"2024-04-26T20:27:51.089370+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:18.835636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.917776+0000","last_change":"2024-04-26T20:27:49.918298+0000","last_active":"2024-04-26T20:27:50.917776+0000","last_peered":"2024-04-26T20:27:50.917776+0000","last_clean":"2024-04-26T20:27:50.917776+0000","last_became_active":"2024-04-26T20:27:49.912487+0000","last_became_peered":"2024-04-26T20:27:49.912487+0000","last_unstale":"2024-04-26T20:27:50.917776+0000","last_undegraded":"2024-04-26T20:27:50.917776+0000","last_fullsized":"2024-04-26T20:27:50.917776+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:25.991117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919131+0000","last_change":"2024-04-26T20:27:49.914198+0000","last_active":"2024-04-26T20:27:50.919131+0000","last_peered":"2024-04-26T20:27:50.919131+0000","last_clean":"2024-04-26T20:27:50.919131+0000","last_became_active":"2024-04-26T20:27:49.913905+0000","last_became_peered":"2024-04-26T20:27:49.913905+0000","last_unstale":"2024-04-26T20:27:50.919131+0000","last_undegraded":"2024-04-26T20:27:50.919131+0000","last_fullsized":"2024-04-26T20:27:50.919131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919693+0000","last_change":"2024-04-26T20:27:49.929062+0000","last_active":"2024-04-26T20:27:50.919693+0000","last_peered":"2024-04-26T20:27:50.919693+0000","last_clean":"2024-04-26T20:27:50.919693+0000","last_became_active":"2024-04-26T20:27:49.926877+0000","last_became_peered":"2024-04-26T20:27:49.926877+0000","last_unstale":"2024-04-26T20:27:50.919693+0000","last_undegraded":"2024-04-26T20:27:50.919693+0000","last_fullsized":"2024-04-26T20:27:50.919693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:39.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088311+0000","last_change":"2024-04-26T20:27:49.899934+0000","last_active":"2024-04-26T20:27:51.088311+0000","last_peered":"2024-04-26T20:27:51.088311+0000","last_clean":"2024-04-26T20:27:51.088311+0000","last_became_active":"2024-04-26T20:27:49.899653+0000","last_became_peered":"2024-04-26T20:27:49.899653+0000","last_unstale":"2024-04-26T20:27:51.088311+0000","last_undegraded":"2024-04-26T20:27:51.088311+0000","last_fullsized":"2024-04-26T20:27:51.088311+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.920464+0000","last_change":"2024-04-26T20:27:49.941773+0000","last_active":"2024-04-26T20:27:50.920464+0000","last_peered":"2024-04-26T20:27:50.920464+0000","last_clean":"2024-04-26T20:27:50.920464+0000","last_became_active":"2024-04-26T20:27:49.941486+0000","last_became_peered":"2024-04-26T20:27:49.941486+0000","last_unstale":"2024-04-26T20:27:50.920464+0000","last_undegraded":"2024-04-26T20:27:50.920464+0000","last_fullsized":"2024-04-26T20:27:50.920464+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.625061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918855+0000","last_change":"2024-04-26T20:27:48.897659+0000","last_active":"2024-04-26T20:27:50.918855+0000","last_peered":"2024-04-26T20:27:50.918855+0000","last_clean":"2024-04-26T20:27:50.918855+0000","last_became_active":"2024-04-26T20:27:48.896450+0000","last_became_peered":"2024-04-26T20:27:48.896450+0000","last_unstale":"2024-04-26T20:27:50.918855+0000","last_undegraded":"2024-04-26T20:27:50.918855+0000","last_fullsized":"2024-04-26T20:27:50.918855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089362+0000","last_change":"2024-04-26T20:27:49.925720+0000","last_active":"2024-04-26T20:27:51.089362+0000","last_peered":"2024-04-26T20:27:51.089362+0000","last_clean":"2024-04-26T20:27:51.089362+0000","last_became_active":"2024-04-26T20:27:49.925062+0000","last_became_peered":"2024-04-26T20:27:49.925062+0000","last_unstale":"2024-04-26T20:27:51.089362+0000","last_undegraded":"2024-04-26T20:27:51.089362+0000","last_fullsized":"2024-04-26T20:27:51.089362+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088786+0000","last_change":"2024-04-26T20:27:48.895441+0000","last_active":"2024-04-26T20:27:51.088786+0000","last_peered":"2024-04-26T20:27:51.088786+0000","last_clean":"2024-04-26T20:27:51.088786+0000","last_became_active":"2024-04-26T20:27:48.895118+0000","last_became_peered":"2024-04-26T20:27:48.895118+0000","last_unstale":"2024-04-26T20:27:51.088786+0000","last_undegraded":"2024-04-26T20:27:51.088786+0000","last_fullsized":"2024-04-26T20:27:51.088786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.092028+0000","last_change":"2024-04-26T20:27:49.928962+0000","last_active":"2024-04-26T20:27:51.092028+0000","last_peered":"2024-04-26T20:27:51.092028+0000","last_clean":"2024-04-26T20:27:51.092028+0000","last_became_active":"2024-04-26T20:27:49.928657+0000","last_became_peered":"2024-04-26T20:27:49.928657+0000","last_unstale":"2024-04-26T20:27:51.092028+0000","last_undegraded":"2024-04-26T20:27:51.092028+0000","last_fullsized":"2024-04-26T20:27:51.092028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089608+0000","last_change":"2024-04-26T20:27:48.894724+0000","last_active":"2024-04-26T20:27:51.089608+0000","last_peered":"2024-04-26T20:27:51.089608+0000","last_clean":"2024-04-26T20:27:51.089608+0000","last_became_active":"2024-04-26T20:27:48.893381+0000","last_became_peered":"2024-04-26T20:27:48.893381+0000","last_unstale":"2024-04-26T20:27:51.089608+0000","last_undegraded":"2024-04-26T20:27:51.089608+0000","last_fullsized":"2024-04-26T20:27:51.089608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091775+0000","last_change":"2024-04-26T20:27:49.929224+0000","last_active":"2024-04-26T20:27:51.091775+0000","last_peered":"2024-04-26T20:27:51.091775+0000","last_clean":"2024-04-26T20:27:51.091775+0000","last_became_active":"2024-04-26T20:27:49.928934+0000","last_became_peered":"2024-04-26T20:27:49.928934+0000","last_unstale":"2024-04-26T20:27:51.091775+0000","last_undegraded":"2024-04-26T20:27:51.091775+0000","last_fullsized":"2024-04-26T20:27:51.091775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:17.876179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091259+0000","last_change":"2024-04-26T20:27:48.894846+0000","last_active":"2024-04-26T20:27:51.091259+0000","last_peered":"2024-04-26T20:27:51.091259+0000","last_clean":"2024-04-26T20:27:51.091259+0000","last_became_active":"2024-04-26T20:27:48.893698+0000","last_became_peered":"2024-04-26T20:27:48.893698+0000","last_unstale":"2024-04-26T20:27:51.091259+0000","last_undegraded":"2024-04-26T20:27:51.091259+0000","last_fullsized":"2024-04-26T20:27:51.091259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:49.845649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.087451+0000","last_change":"2024-04-26T20:27:49.920847+0000","last_active":"2024-04-26T20:27:51.087451+0000","last_peered":"2024-04-26T20:27:51.087451+0000","last_clean":"2024-04-26T20:27:51.087451+0000","last_became_active":"2024-04-26T20:27:49.920522+0000","last_became_peered":"2024-04-26T20:27:49.920522+0000","last_unstale":"2024-04-26T20:27:51.087451+0000","last_undegraded":"2024-04-26T20:27:51.087451+0000","last_fullsized":"2024-04-26T20:27:51.087451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:27.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089223+0000","last_change":"2024-04-26T20:27:48.891209+0000","last_active":"2024-04-26T20:27:51.089223+0000","last_peered":"2024-04-26T20:27:51.089223+0000","last_clean":"2024-04-26T20:27:51.089223+0000","last_became_active":"2024-04-26T20:27:48.890894+0000","last_became_peered":"2024-04-26T20:27:48.890894+0000","last_unstale":"2024-04-26T20:27:51.089223+0000","last_undegraded":"2024-04-26T20:27:51.089223+0000","last_fullsized":"2024-04-26T20:27:51.089223+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089249+0000","last_change":"2024-04-26T20:27:49.920288+0000","last_active":"2024-04-26T20:27:51.089249+0000","last_peered":"2024-04-26T20:27:51.089249+0000","last_clean":"2024-04-26T20:27:51.089249+0000","last_became_active":"2024-04-26T20:27:49.919780+0000","last_became_peered":"2024-04-26T20:27:49.919780+0000","last_unstale":"2024-04-26T20:27:51.089249+0000","last_undegraded":"2024-04-26T20:27:51.089249+0000","last_fullsized":"2024-04-26T20:27:51.089249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:52.868375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919697+0000","last_change":"2024-04-26T20:27:48.897588+0000","last_active":"2024-04-26T20:27:50.919697+0000","last_peered":"2024-04-26T20:27:50.919697+0000","last_clean":"2024-04-26T20:27:50.919697+0000","last_became_active":"2024-04-26T20:27:48.896574+0000","last_became_peered":"2024-04-26T20:27:48.896574+0000","last_unstale":"2024-04-26T20:27:50.919697+0000","last_undegraded":"2024-04-26T20:27:50.919697+0000","last_fullsized":"2024-04-26T20:27:50.919697+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:22.623531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091842+0000","last_change":"2024-04-26T20:27:49.916775+0000","last_active":"2024-04-26T20:27:51.091842+0000","last_peered":"2024-04-26T20:27:51.091842+0000","last_clean":"2024-04-26T20:27:51.091842+0000","last_became_active":"2024-04-26T20:27:49.916312+0000","last_became_peered":"2024-04-26T20:27:49.916312+0000","last_unstale":"2024-04-26T20:27:51.091842+0000","last_undegraded":"2024-04-26T20:27:51.091842+0000","last_fullsized":"2024-04-26T20:27:51.091842+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:18.815083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088126+0000","last_change":"2024-04-26T20:27:48.898227+0000","last_active":"2024-04-26T20:27:51.088126+0000","last_peered":"2024-04-26T20:27:51.088126+0000","last_clean":"2024-04-26T20:27:51.088126+0000","last_became_active":"2024-04-26T20:27:48.897196+0000","last_became_peered":"2024-04-26T20:27:48.897196+0000","last_unstale":"2024-04-26T20:27:51.088126+0000","last_undegraded":"2024-04-26T20:27:51.088126+0000","last_fullsized":"2024-04-26T20:27:51.088126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:46.834398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090571+0000","last_change":"2024-04-26T20:27:49.933505+0000","last_active":"2024-04-26T20:27:51.090571+0000","last_peered":"2024-04-26T20:27:51.090571+0000","last_clean":"2024-04-26T20:27:51.090571+0000","last_became_active":"2024-04-26T20:27:49.933145+0000","last_became_peered":"2024-04-26T20:27:49.933145+0000","last_unstale":"2024-04-26T20:27:51.090571+0000","last_undegraded":"2024-04-26T20:27:51.090571+0000","last_fullsized":"2024-04-26T20:27:51.090571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:47.609089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090388+0000","last_change":"2024-04-26T20:27:48.890830+0000","last_active":"2024-04-26T20:27:51.090388+0000","last_peered":"2024-04-26T20:27:51.090388+0000","last_clean":"2024-04-26T20:27:51.090388+0000","last_became_active":"2024-04-26T20:27:48.890544+0000","last_became_peered":"2024-04-26T20:27:48.890544+0000","last_unstale":"2024-04-26T20:27:51.090388+0000","last_undegraded":"2024-04-26T20:27:51.090388+0000","last_fullsized":"2024-04-26T20:27:51.090388+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913343+0000","last_change":"2024-04-26T20:27:49.923399+0000","last_active":"2024-04-26T20:27:50.913343+0000","last_peered":"2024-04-26T20:27:50.913343+0000","last_clean":"2024-04-26T20:27:50.913343+0000","last_became_active":"2024-04-26T20:27:49.923127+0000","last_became_peered":"2024-04-26T20:27:49.923127+0000","last_unstale":"2024-04-26T20:27:50.913343+0000","last_undegraded":"2024-04-26T20:27:50.913343+0000","last_fullsized":"2024-04-26T20:27:50.913343+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:20.075848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091883+0000","last_change":"2024-04-26T20:27:48.889003+0000","last_active":"2024-04-26T20:27:51.091883+0000","last_peered":"2024-04-26T20:27:51.091883+0000","last_clean":"2024-04-26T20:27:51.091883+0000","last_became_active":"2024-04-26T20:27:48.888708+0000","last_became_peered":"2024-04-26T20:27:48.888708+0000","last_unstale":"2024-04-26T20:27:51.091883+0000","last_undegraded":"2024-04-26T20:27:51.091883+0000","last_fullsized":"2024-04-26T20:27:51.091883+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:17.802067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.920551+0000","last_change":"2024-04-26T20:27:49.939100+0000","last_active":"2024-04-26T20:27:50.920551+0000","last_peered":"2024-04-26T20:27:50.920551+0000","last_clean":"2024-04-26T20:27:50.920551+0000","last_became_active":"2024-04-26T20:27:49.938778+0000","last_became_peered":"2024-04-26T20:27:49.938778+0000","last_unstale":"2024-04-26T20:27:50.920551+0000","last_undegraded":"2024-04-26T20:27:50.920551+0000","last_fullsized":"2024-04-26T20:27:50.920551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:18.835636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089588+0000","last_change":"2024-04-26T20:27:48.897655+0000","last_active":"2024-04-26T20:27:51.089588+0000","last_peered":"2024-04-26T20:27:51.089588+0000","last_clean":"2024-04-26T20:27:51.089588+0000","last_became_active":"2024-04-26T20:27:48.897232+0000","last_became_peered":"2024-04-26T20:27:48.897232+0000","last_unstale":"2024-04-26T20:27:51.089588+0000","last_undegraded":"2024-04-26T20:27:51.089588+0000","last_fullsized":"2024-04-26T20:27:51.089588+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088562+0000","last_change":"2024-04-26T20:27:49.921057+0000","last_active":"2024-04-26T20:27:51.088562+0000","last_peered":"2024-04-26T20:27:51.088562+0000","last_clean":"2024-04-26T20:27:51.088562+0000","last_became_active":"2024-04-26T20:27:49.920747+0000","last_became_peered":"2024-04-26T20:27:49.920747+0000","last_unstale":"2024-04-26T20:27:51.088562+0000","last_undegraded":"2024-04-26T20:27:51.088562+0000","last_fullsized":"2024-04-26T20:27:51.088562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:02:24.557657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089743+0000","last_change":"2024-04-26T20:27:48.895154+0000","last_active":"2024-04-26T20:27:51.089743+0000","last_peered":"2024-04-26T20:27:51.089743+0000","last_clean":"2024-04-26T20:27:51.089743+0000","last_became_active":"2024-04-26T20:27:48.894300+0000","last_became_peered":"2024-04-26T20:27:48.894300+0000","last_unstale":"2024-04-26T20:27:51.089743+0000","last_undegraded":"2024-04-26T20:27:51.089743+0000","last_fullsized":"2024-04-26T20:27:51.089743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091627+0000","last_change":"2024-04-26T20:27:49.920045+0000","last_active":"2024-04-26T20:27:51.091627+0000","last_peered":"2024-04-26T20:27:51.091627+0000","last_clean":"2024-04-26T20:27:51.091627+0000","last_became_active":"2024-04-26T20:27:49.919739+0000","last_became_peered":"2024-04-26T20:27:49.919739+0000","last_unstale":"2024-04-26T20:27:51.091627+0000","last_undegraded":"2024-04-26T20:27:51.091627+0000","last_fullsized":"2024-04-26T20:27:51.091627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:51.076713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091205+0000","last_change":"2024-04-26T20:27:48.896675+0000","last_active":"2024-04-26T20:27:51.091205+0000","last_peered":"2024-04-26T20:27:51.091205+0000","last_clean":"2024-04-26T20:27:51.091205+0000","last_became_active":"2024-04-26T20:27:48.896362+0000","last_became_peered":"2024-04-26T20:27:48.896362+0000","last_unstale":"2024-04-26T20:27:51.091205+0000","last_undegraded":"2024-04-26T20:27:51.091205+0000","last_fullsized":"2024-04-26T20:27:51.091205+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:39.374969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089278+0000","last_change":"2024-04-26T20:27:49.920198+0000","last_active":"2024-04-26T20:27:51.089278+0000","last_peered":"2024-04-26T20:27:51.089278+0000","last_clean":"2024-04-26T20:27:51.089278+0000","last_became_active":"2024-04-26T20:27:49.919936+0000","last_became_peered":"2024-04-26T20:27:49.919936+0000","last_unstale":"2024-04-26T20:27:51.089278+0000","last_undegraded":"2024-04-26T20:27:51.089278+0000","last_fullsized":"2024-04-26T20:27:51.089278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:16.234837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090800+0000","last_change":"2024-04-26T20:27:48.891091+0000","last_active":"2024-04-26T20:27:51.090800+0000","last_peered":"2024-04-26T20:27:51.090800+0000","last_clean":"2024-04-26T20:27:51.090800+0000","last_became_active":"2024-04-26T20:27:48.888052+0000","last_became_peered":"2024-04-26T20:27:48.888052+0000","last_unstale":"2024-04-26T20:27:51.090800+0000","last_undegraded":"2024-04-26T20:27:51.090800+0000","last_fullsized":"2024-04-26T20:27:51.090800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:35.082176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091638+0000","last_change":"2024-04-26T20:27:49.917196+0000","last_active":"2024-04-26T20:27:51.091638+0000","last_peered":"2024-04-26T20:27:51.091638+0000","last_clean":"2024-04-26T20:27:51.091638+0000","last_became_active":"2024-04-26T20:27:49.916941+0000","last_became_peered":"2024-04-26T20:27:49.916941+0000","last_unstale":"2024-04-26T20:27:51.091638+0000","last_undegraded":"2024-04-26T20:27:51.091638+0000","last_fullsized":"2024-04-26T20:27:51.091638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:55.419963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.087490+0000","last_change":"2024-04-26T20:27:48.894730+0000","last_active":"2024-04-26T20:27:51.087490+0000","last_peered":"2024-04-26T20:27:51.087490+0000","last_clean":"2024-04-26T20:27:51.087490+0000","last_became_active":"2024-04-26T20:27:48.894446+0000","last_became_peered":"2024-04-26T20:27:48.894446+0000","last_unstale":"2024-04-26T20:27:51.087490+0000","last_undegraded":"2024-04-26T20:27:51.087490+0000","last_fullsized":"2024-04-26T20:27:51.087490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.695947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919733+0000","last_change":"2024-04-26T20:27:49.928141+0000","last_active":"2024-04-26T20:27:50.919733+0000","last_peered":"2024-04-26T20:27:50.919733+0000","last_clean":"2024-04-26T20:27:50.919733+0000","last_became_active":"2024-04-26T20:27:49.926664+0000","last_became_peered":"2024-04-26T20:27:49.926664+0000","last_unstale":"2024-04-26T20:27:50.919733+0000","last_undegraded":"2024-04-26T20:27:50.919733+0000","last_fullsized":"2024-04-26T20:27:50.919733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:47.609089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.917183+0000","last_change":"2024-04-26T20:27:48.886899+0000","last_active":"2024-04-26T20:27:50.917183+0000","last_peered":"2024-04-26T20:27:50.917183+0000","last_clean":"2024-04-26T20:27:50.917183+0000","last_became_active":"2024-04-26T20:27:48.886536+0000","last_became_peered":"2024-04-26T20:27:48.886536+0000","last_unstale":"2024-04-26T20:27:50.917183+0000","last_undegraded":"2024-04-26T20:27:50.917183+0000","last_fullsized":"2024-04-26T20:27:50.917183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:01.561560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090504+0000","last_change":"2024-04-26T20:27:49.918902+0000","last_active":"2024-04-26T20:27:51.090504+0000","last_peered":"2024-04-26T20:27:51.090504+0000","last_clean":"2024-04-26T20:27:51.090504+0000","last_became_active":"2024-04-26T20:27:49.918340+0000","last_became_peered":"2024-04-26T20:27:49.918340+0000","last_unstale":"2024-04-26T20:27:51.090504+0000","last_undegraded":"2024-04-26T20:27:51.090504+0000","last_fullsized":"2024-04-26T20:27:51.090504+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:39.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.917839+0000","last_change":"2024-04-26T20:27:48.886522+0000","last_active":"2024-04-26T20:27:50.917839+0000","last_peered":"2024-04-26T20:27:50.917839+0000","last_clean":"2024-04-26T20:27:50.917839+0000","last_became_active":"2024-04-26T20:27:48.885874+0000","last_became_peered":"2024-04-26T20:27:48.885874+0000","last_unstale":"2024-04-26T20:27:50.917839+0000","last_undegraded":"2024-04-26T20:27:50.917839+0000","last_fullsized":"2024-04-26T20:27:50.917839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:12:24.220385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091232+0000","last_change":"2024-04-26T20:27:49.926821+0000","last_active":"2024-04-26T20:27:51.091232+0000","last_peered":"2024-04-26T20:27:51.091232+0000","last_clean":"2024-04-26T20:27:51.091232+0000","last_became_active":"2024-04-26T20:27:49.926564+0000","last_became_peered":"2024-04-26T20:27:49.926564+0000","last_unstale":"2024-04-26T20:27:51.091232+0000","last_undegraded":"2024-04-26T20:27:51.091232+0000","last_fullsized":"2024-04-26T20:27:51.091232+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:43:18.835636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089800+0000","last_change":"2024-04-26T20:27:48.900526+0000","last_active":"2024-04-26T20:27:51.089800+0000","last_peered":"2024-04-26T20:27:51.089800+0000","last_clean":"2024-04-26T20:27:51.089800+0000","last_became_active":"2024-04-26T20:27:48.900110+0000","last_became_peered":"2024-04-26T20:27:48.900110+0000","last_unstale":"2024-04-26T20:27:51.089800+0000","last_undegraded":"2024-04-26T20:27:51.089800+0000","last_fullsized":"2024-04-26T20:27:51.089800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:09:26.369631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089933+0000","last_change":"2024-04-26T20:27:49.923959+0000","last_active":"2024-04-26T20:27:51.089933+0000","last_peered":"2024-04-26T20:27:51.089933+0000","last_clean":"2024-04-26T20:27:51.089933+0000","last_became_active":"2024-04-26T20:27:49.923064+0000","last_became_peered":"2024-04-26T20:27:49.923064+0000","last_unstale":"2024-04-26T20:27:51.089933+0000","last_undegraded":"2024-04-26T20:27:51.089933+0000","last_fullsized":"2024-04-26T20:27:51.089933+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:27.969586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.092144+0000","last_change":"2024-04-26T20:27:48.899650+0000","last_active":"2024-04-26T20:27:51.092144+0000","last_peered":"2024-04-26T20:27:51.092144+0000","last_clean":"2024-04-26T20:27:51.092144+0000","last_became_active":"2024-04-26T20:27:48.899036+0000","last_became_peered":"2024-04-26T20:27:48.899036+0000","last_unstale":"2024-04-26T20:27:51.092144+0000","last_undegraded":"2024-04-26T20:27:51.092144+0000","last_fullsized":"2024-04-26T20:27:51.092144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:33.237276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091840+0000","last_change":"2024-04-26T20:27:49.901218+0000","last_active":"2024-04-26T20:27:51.091840+0000","last_peered":"2024-04-26T20:27:51.091840+0000","last_clean":"2024-04-26T20:27:51.091840+0000","last_became_active":"2024-04-26T20:27:49.900732+0000","last_became_peered":"2024-04-26T20:27:49.900732+0000","last_unstale":"2024-04-26T20:27:51.091840+0000","last_undegraded":"2024-04-26T20:27:51.091840+0000","last_fullsized":"2024-04-26T20:27:51.091840+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:36.095191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913413+0000","last_change":"2024-04-26T20:27:48.883412+0000","last_active":"2024-04-26T20:27:50.913413+0000","last_peered":"2024-04-26T20:27:50.913413+0000","last_clean":"2024-04-26T20:27:50.913413+0000","last_became_active":"2024-04-26T20:27:48.882939+0000","last_became_peered":"2024-04-26T20:27:48.882939+0000","last_unstale":"2024-04-26T20:27:50.913413+0000","last_undegraded":"2024-04-26T20:27:50.913413+0000","last_fullsized":"2024-04-26T20:27:50.913413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.174009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913319+0000","last_change":"2024-04-26T20:27:49.909503+0000","last_active":"2024-04-26T20:27:50.913319+0000","last_peered":"2024-04-26T20:27:50.913319+0000","last_clean":"2024-04-26T20:27:50.913319+0000","last_became_active":"2024-04-26T20:27:49.909122+0000","last_became_peered":"2024-04-26T20:27:49.909122+0000","last_unstale":"2024-04-26T20:27:50.913319+0000","last_undegraded":"2024-04-26T20:27:50.913319+0000","last_fullsized":"2024-04-26T20:27:50.913319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:50.858665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089634+0000","last_change":"2024-04-26T20:27:48.885014+0000","last_active":"2024-04-26T20:27:51.089634+0000","last_peered":"2024-04-26T20:27:51.089634+0000","last_clean":"2024-04-26T20:27:51.089634+0000","last_became_active":"2024-04-26T20:27:48.884727+0000","last_became_peered":"2024-04-26T20:27:48.884727+0000","last_unstale":"2024-04-26T20:27:51.089634+0000","last_undegraded":"2024-04-26T20:27:51.089634+0000","last_fullsized":"2024-04-26T20:27:51.089634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918556+0000","last_change":"2024-04-26T20:27:49.932590+0000","last_active":"2024-04-26T20:27:50.918556+0000","last_peered":"2024-04-26T20:27:50.918556+0000","last_clean":"2024-04-26T20:27:50.918556+0000","last_became_active":"2024-04-26T20:27:49.931344+0000","last_became_peered":"2024-04-26T20:27:49.931344+0000","last_unstale":"2024-04-26T20:27:50.918556+0000","last_undegraded":"2024-04-26T20:27:50.918556+0000","last_fullsized":"2024-04-26T20:27:50.918556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.912998+0000","last_change":"2024-04-26T20:27:48.880032+0000","last_active":"2024-04-26T20:27:50.912998+0000","last_peered":"2024-04-26T20:27:50.912998+0000","last_clean":"2024-04-26T20:27:50.912998+0000","last_became_active":"2024-04-26T20:27:48.878893+0000","last_became_peered":"2024-04-26T20:27:48.878893+0000","last_unstale":"2024-04-26T20:27:50.912998+0000","last_undegraded":"2024-04-26T20:27:50.912998+0000","last_fullsized":"2024-04-26T20:27:50.912998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:04.775515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088856+0000","last_change":"2024-04-26T20:27:49.927225+0000","last_active":"2024-04-26T20:27:51.088856+0000","last_peered":"2024-04-26T20:27:51.088856+0000","last_clean":"2024-04-26T20:27:51.088856+0000","last_became_active":"2024-04-26T20:27:49.925074+0000","last_became_peered":"2024-04-26T20:27:49.925074+0000","last_unstale":"2024-04-26T20:27:51.088856+0000","last_undegraded":"2024-04-26T20:27:51.088856+0000","last_fullsized":"2024-04-26T20:27:51.088856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:08.741667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.912968+0000","last_change":"2024-04-26T20:27:48.896812+0000","last_active":"2024-04-26T20:27:50.912968+0000","last_peered":"2024-04-26T20:27:50.912968+0000","last_clean":"2024-04-26T20:27:50.912968+0000","last_became_active":"2024-04-26T20:27:48.895863+0000","last_became_peered":"2024-04-26T20:27:48.895863+0000","last_unstale":"2024-04-26T20:27:50.912968+0000","last_undegraded":"2024-04-26T20:27:50.912968+0000","last_fullsized":"2024-04-26T20:27:50.912968+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:23:43.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088640+0000","last_change":"2024-04-26T20:27:49.927088+0000","last_active":"2024-04-26T20:27:51.088640+0000","last_peered":"2024-04-26T20:27:51.088640+0000","last_clean":"2024-04-26T20:27:51.088640+0000","last_became_active":"2024-04-26T20:27:49.926795+0000","last_became_peered":"2024-04-26T20:27:49.926795+0000","last_unstale":"2024-04-26T20:27:51.088640+0000","last_undegraded":"2024-04-26T20:27:51.088640+0000","last_fullsized":"2024-04-26T20:27:51.088640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:39.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.919970+0000","last_change":"2024-04-26T20:27:48.891475+0000","last_active":"2024-04-26T20:27:50.919970+0000","last_peered":"2024-04-26T20:27:50.919970+0000","last_clean":"2024-04-26T20:27:50.919970+0000","last_became_active":"2024-04-26T20:27:48.891058+0000","last_became_peered":"2024-04-26T20:27:48.891058+0000","last_unstale":"2024-04-26T20:27:50.919970+0000","last_undegraded":"2024-04-26T20:27:50.919970+0000","last_fullsized":"2024-04-26T20:27:50.919970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:23.923671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090695+0000","last_change":"2024-04-26T20:27:49.921492+0000","last_active":"2024-04-26T20:27:51.090695+0000","last_peered":"2024-04-26T20:27:51.090695+0000","last_clean":"2024-04-26T20:27:51.090695+0000","last_became_active":"2024-04-26T20:27:49.921208+0000","last_became_peered":"2024-04-26T20:27:49.921208+0000","last_unstale":"2024-04-26T20:27:51.090695+0000","last_undegraded":"2024-04-26T20:27:51.090695+0000","last_fullsized":"2024-04-26T20:27:51.090695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:30.625061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.912808+0000","last_change":"2024-04-26T20:27:48.879281+0000","last_active":"2024-04-26T20:27:50.912808+0000","last_peered":"2024-04-26T20:27:50.912808+0000","last_clean":"2024-04-26T20:27:50.912808+0000","last_became_active":"2024-04-26T20:27:48.878964+0000","last_became_peered":"2024-04-26T20:27:48.878964+0000","last_unstale":"2024-04-26T20:27:50.912808+0000","last_undegraded":"2024-04-26T20:27:50.912808+0000","last_fullsized":"2024-04-26T20:27:50.912808+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:32.854744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088661+0000","last_change":"2024-04-26T20:27:49.925812+0000","last_active":"2024-04-26T20:27:51.088661+0000","last_peered":"2024-04-26T20:27:51.088661+0000","last_clean":"2024-04-26T20:27:51.088661+0000","last_became_active":"2024-04-26T20:27:49.924910+0000","last_became_peered":"2024-04-26T20:27:49.924910+0000","last_unstale":"2024-04-26T20:27:51.088661+0000","last_undegraded":"2024-04-26T20:27:51.088661+0000","last_fullsized":"2024-04-26T20:27:51.088661+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:53.766630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090094+0000","last_change":"2024-04-26T20:27:48.896599+0000","last_active":"2024-04-26T20:27:51.090094+0000","last_peered":"2024-04-26T20:27:51.090094+0000","last_clean":"2024-04-26T20:27:51.090094+0000","last_became_active":"2024-04-26T20:27:48.896224+0000","last_became_peered":"2024-04-26T20:27:48.896224+0000","last_unstale":"2024-04-26T20:27:51.090094+0000","last_undegraded":"2024-04-26T20:27:51.090094+0000","last_fullsized":"2024-04-26T20:27:51.090094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:52.021385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.921041+0000","last_change":"2024-04-26T20:27:49.937124+0000","last_active":"2024-04-26T20:27:50.921041+0000","last_peered":"2024-04-26T20:27:50.921041+0000","last_clean":"2024-04-26T20:27:50.921041+0000","last_became_active":"2024-04-26T20:27:49.936510+0000","last_became_peered":"2024-04-26T20:27:49.936510+0000","last_unstale":"2024-04-26T20:27:50.921041+0000","last_undegraded":"2024-04-26T20:27:50.921041+0000","last_fullsized":"2024-04-26T20:27:50.921041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:41:38.269345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.913247+0000","last_change":"2024-04-26T20:27:48.892674+0000","last_active":"2024-04-26T20:27:50.913247+0000","last_peered":"2024-04-26T20:27:50.913247+0000","last_clean":"2024-04-26T20:27:50.913247+0000","last_became_active":"2024-04-26T20:27:48.892364+0000","last_became_peered":"2024-04-26T20:27:48.892364+0000","last_unstale":"2024-04-26T20:27:50.913247+0000","last_undegraded":"2024-04-26T20:27:50.913247+0000","last_fullsized":"2024-04-26T20:27:50.913247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:56.403870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.091399+0000","last_change":"2024-04-26T20:27:49.918395+0000","last_active":"2024-04-26T20:27:51.091399+0000","last_peered":"2024-04-26T20:27:51.091399+0000","last_clean":"2024-04-26T20:27:51.091399+0000","last_became_active":"2024-04-26T20:27:49.918111+0000","last_became_peered":"2024-04-26T20:27:49.918111+0000","last_unstale":"2024-04-26T20:27:51.091399+0000","last_undegraded":"2024-04-26T20:27:51.091399+0000","last_fullsized":"2024-04-26T20:27:51.091399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:41:38.269345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088315+0000","last_change":"2024-04-26T20:27:48.886260+0000","last_active":"2024-04-26T20:27:51.088315+0000","last_peered":"2024-04-26T20:27:51.088315+0000","last_clean":"2024-04-26T20:27:51.088315+0000","last_became_active":"2024-04-26T20:27:48.885935+0000","last_became_peered":"2024-04-26T20:27:48.885935+0000","last_unstale":"2024-04-26T20:27:51.088315+0000","last_undegraded":"2024-04-26T20:27:51.088315+0000","last_fullsized":"2024-04-26T20:27:51.088315+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:12:24.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090018+0000","last_change":"2024-04-26T20:27:49.919086+0000","last_active":"2024-04-26T20:27:51.090018+0000","last_peered":"2024-04-26T20:27:51.090018+0000","last_clean":"2024-04-26T20:27:51.090018+0000","last_became_active":"2024-04-26T20:27:49.913716+0000","last_became_peered":"2024-04-26T20:27:49.913716+0000","last_unstale":"2024-04-26T20:27:51.090018+0000","last_undegraded":"2024-04-26T20:27:51.090018+0000","last_fullsized":"2024-04-26T20:27:51.090018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:12:44.003646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.088297+0000","last_change":"2024-04-26T20:27:48.892408+0000","last_active":"2024-04-26T20:27:51.088297+0000","last_peered":"2024-04-26T20:27:51.088297+0000","last_clean":"2024-04-26T20:27:51.088297+0000","last_became_active":"2024-04-26T20:27:48.891998+0000","last_became_peered":"2024-04-26T20:27:48.891998+0000","last_unstale":"2024-04-26T20:27:51.088297+0000","last_undegraded":"2024-04-26T20:27:51.088297+0000","last_fullsized":"2024-04-26T20:27:51.088297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:07.661728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.087855+0000","last_change":"2024-04-26T20:27:49.912958+0000","last_active":"2024-04-26T20:27:51.087855+0000","last_peered":"2024-04-26T20:27:51.087855+0000","last_clean":"2024-04-26T20:27:51.087855+0000","last_became_active":"2024-04-26T20:27:49.912656+0000","last_became_peered":"2024-04-26T20:27:49.912656+0000","last_unstale":"2024-04-26T20:27:51.087855+0000","last_undegraded":"2024-04-26T20:27:51.087855+0000","last_fullsized":"2024-04-26T20:27:51.087855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:57.725728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918135+0000","last_change":"2024-04-26T20:27:48.883806+0000","last_active":"2024-04-26T20:27:50.918135+0000","last_peered":"2024-04-26T20:27:50.918135+0000","last_clean":"2024-04-26T20:27:50.918135+0000","last_became_active":"2024-04-26T20:27:48.883495+0000","last_became_peered":"2024-04-26T20:27:48.883495+0000","last_unstale":"2024-04-26T20:27:50.918135+0000","last_undegraded":"2024-04-26T20:27:50.918135+0000","last_fullsized":"2024-04-26T20:27:50.918135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090728+0000","last_change":"2024-04-26T20:27:49.907230+0000","last_active":"2024-04-26T20:27:51.090728+0000","last_peered":"2024-04-26T20:27:51.090728+0000","last_clean":"2024-04-26T20:27:51.090728+0000","last_became_active":"2024-04-26T20:27:49.905877+0000","last_became_peered":"2024-04-26T20:27:49.905877+0000","last_unstale":"2024-04-26T20:27:51.090728+0000","last_undegraded":"2024-04-26T20:27:51.090728+0000","last_fullsized":"2024-04-26T20:27:51.090728+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:23.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090287+0000","last_change":"2024-04-26T20:27:48.893801+0000","last_active":"2024-04-26T20:27:51.090287+0000","last_peered":"2024-04-26T20:27:51.090287+0000","last_clean":"2024-04-26T20:27:51.090287+0000","last_became_active":"2024-04-26T20:27:48.893424+0000","last_became_peered":"2024-04-26T20:27:48.893424+0000","last_unstale":"2024-04-26T20:27:51.090287+0000","last_undegraded":"2024-04-26T20:27:51.090287+0000","last_fullsized":"2024-04-26T20:27:51.090287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:19.062832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.920511+0000","last_change":"2024-04-26T20:27:49.936414+0000","last_active":"2024-04-26T20:27:50.920511+0000","last_peered":"2024-04-26T20:27:50.920511+0000","last_clean":"2024-04-26T20:27:50.920511+0000","last_became_active":"2024-04-26T20:27:49.935509+0000","last_became_peered":"2024-04-26T20:27:49.935509+0000","last_unstale":"2024-04-26T20:27:50.920511+0000","last_undegraded":"2024-04-26T20:27:50.920511+0000","last_fullsized":"2024-04-26T20:27:50.920511+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:52.868375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.090711+0000","last_change":"2024-04-26T20:27:48.885205+0000","last_active":"2024-04-26T20:27:51.090711+0000","last_peered":"2024-04-26T20:27:51.090711+0000","last_clean":"2024-04-26T20:27:51.090711+0000","last_became_active":"2024-04-26T20:27:48.884908+0000","last_became_peered":"2024-04-26T20:27:48.884908+0000","last_unstale":"2024-04-26T20:27:51.090711+0000","last_undegraded":"2024-04-26T20:27:51.090711+0000","last_fullsized":"2024-04-26T20:27:51.090711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:51.089668+0000","last_change":"2024-04-26T20:27:49.925449+0000","last_active":"2024-04-26T20:27:51.089668+0000","last_peered":"2024-04-26T20:27:51.089668+0000","last_clean":"2024-04-26T20:27:51.089668+0000","last_became_active":"2024-04-26T20:27:49.925098+0000","last_became_peered":"2024-04-26T20:27:49.925098+0000","last_unstale":"2024-04-26T20:27:51.089668+0000","last_undegraded":"2024-04-26T20:27:51.089668+0000","last_fullsized":"2024-04-26T20:27:51.089668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:48.849301+0000","last_clean_scrub_stamp":"2024-04-26T20:27:48.849301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:41:38.269345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.920445+0000","last_change":"2024-04-26T20:27:48.889736+0000","last_active":"2024-04-26T20:27:50.920445+0000","last_peered":"2024-04-26T20:27:50.920445+0000","last_clean":"2024-04-26T20:27:50.920445+0000","last_became_active":"2024-04-26T20:27:48.889103+0000","last_became_peered":"2024-04-26T20:27:48.889103+0000","last_unstale":"2024-04-26T20:27:50.920445+0000","last_undegraded":"2024-04-26T20:27:50.920445+0000","last_fullsized":"2024-04-26T20:27:50.920445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:27:47.836285+0000","last_clean_scrub_stamp":"2024-04-26T20:27:47.836285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:19:50.063697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T20:27:50.918581+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:27:50.918581+0000","last_peered":"2024-04-26T20:27:50.918581+0000","last_clean":"2024-04-26T20:27:50.918581+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:27:50.918581+0000","last_undegraded":"2024-04-26T20:27:50.918581+0000","last_fullsized":"2024-04-26T20:27:50.918581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":782280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":190834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":191876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":190300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":780632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:28:00.003 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:28:00.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:28:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.262+0000 7f61eb000640 1 Processor -- start 2024-04-26T20:28:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.262+0000 7f61eb000640 1 -- start start 2024-04-26T20:28:00.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61eb000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f61e40d35d0 0x7f61e40d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61eb000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e40d04f0 con 0x7f61e415ee40 2024-04-26T20:28:00.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61eb000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e40d07d0 con 0x7f61e40d57e0 2024-04-26T20:28:00.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61eb000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f61e40d02a0 con 0x7f61e40d35d0 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61e9c00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f61e415ee40 0x7f61e40d2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38976/0 (socket says 172.21.15.43:38976) 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61e9c00640 1 -- 172.21.15.43:0/2069875935 learned_addr learned my addr 172.21.15.43:0/2069875935 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.264+0000 7f61e9200640 1 --2- 172.21.15.43:0/2069875935 >> v2:172.21.15.43:3300/0 conn(0x7f61e40d35d0 0x7f61e40d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.265+0000 7f61e3e00640 1 -- 172.21.15.43:0/2069875935 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750320756 0 0) 0x7f61e40d04f0 con 0x7f61e415ee40 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.265+0000 7f61e3e00640 1 -- 172.21.15.43:0/2069875935 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61cc003520 con 0x7f61e415ee40 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.265+0000 7f61e9200640 1 -- 172.21.15.43:0/2069875935 >> v1:172.21.15.164:6789/0 conn(0x7f61e40d57e0 legacy=0x7f61e40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.265+0000 7f61e9200640 1 -- 172.21.15.43:0/2069875935 >> v1:172.21.15.43:6789/0 conn(0x7f61e415ee40 legacy=0x7f61e40d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.265+0000 7f61e9200640 1 -- 172.21.15.43:0/2069875935 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e40d0ab0 con 0x7f61e40d35d0 2024-04-26T20:28:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.268+0000 7f61e9200640 1 --2- 172.21.15.43:0/2069875935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d35d0 0x7f61e40d2890 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097a0 tx=0x7f61d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2992ad0192ee74e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.268+0000 7f61e3e00640 1 -- 172.21.15.43:0/2069875935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037070 con 0x7f61e40d35d0 2024-04-26T20:28:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.268+0000 7f61e3e00640 1 -- 172.21.15.43:0/2069875935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0032440 con 0x7f61e40d35d0 2024-04-26T20:28:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.268+0000 7f61e3e00640 1 -- 172.21.15.43:0/2069875935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0032810 con 0x7f61e40d35d0 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 -- 172.21.15.43:0/2069875935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d35d0 msgr2=0x7f61e40d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 --2- 172.21.15.43:0/2069875935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d35d0 0x7f61e40d2890 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097a0 tx=0x7f61d002a230 comp rx=0 tx=0).stop 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 -- 172.21.15.43:0/2069875935 shutdown_connections 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 --2- 172.21.15.43:0/2069875935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d35d0 0x7f61e40d2890 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 -- 172.21.15.43:0/2069875935 >> 172.21.15.43:0/2069875935 conn(0x7f61e40d4cc0 msgr2=0x7f61e40d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 -- 172.21.15.43:0/2069875935 shutdown_connections 2024-04-26T20:28:00.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.269+0000 7f61eb000640 1 -- 172.21.15.43:0/2069875935 wait complete. 2024-04-26T20:28:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.271+0000 7f61eb000640 1 Processor -- start 2024-04-26T20:28:00.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.271+0000 7f61eb000640 1 -- start start 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 0x7f61e4161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e415ee40 0x7f61e40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f61e415f220 con 0x7f61e40d57e0 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61e40b4110 con 0x7f61e415ee40 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61eb000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f61e415f3a0 con 0x7f61e40d35d0 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.272+0000 7f61e9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36690/0 (socket says 172.21.15.43:36690) 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 -- 172.21.15.43:0/166599707 learned_addr learned my addr 172.21.15.43:0/166599707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61ea600640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e415ee40 0x7f61e40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9c00640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 0x7f61e4161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 -- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 msgr2=0x7f61e4161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 0x7f61e4161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 -- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e415ee40 msgr2=0x7f61e40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e415ee40 0x7f61e40cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 -- 172.21.15.43:0/166599707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e4160c60 con 0x7f61e40d57e0 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9c00640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 0x7f61e4161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:00.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.273+0000 7f61e9200640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61d00098c0 tx=0x7f61d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.274+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037040 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.274+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0030500 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.274+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00308d0 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.274+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e4160f40 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.274+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e4161500 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.275+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e40506c0 con 0x7f61e40d57e0 2024-04-26T20:28:00.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.276+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61d0044020 con 0x7f61e40d57e0 2024-04-26T20:28:00.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.277+0000 7f61e1600640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 0x7f61c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.277+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f61d00abf70 con 0x7f61e40d57e0 2024-04-26T20:28:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.277+0000 7f61e9c00640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 0x7f61c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.278+0000 7f61e9c00640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 0x7f61c406ee00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61d80011e0 tx=0x7f61d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.282+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61d007a1f0 con 0x7f61e40d57e0 2024-04-26T20:28:00.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.562+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f61e40b5310 con 0x7f61e40d57e0 2024-04-26T20:28:00.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.567+0000 7f61e1600640 1 -- 172.21.15.43:0/166599707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f61d0079af0 con 0x7f61e40d57e0 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCcDSxmFcmKMxAAcYLYOImoPh7zyitYMx2TqQ== 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:28:00.569 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 msgr2=0x7f61c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 0x7f61c406ee00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f61d80011e0 tx=0x7f61d8009040 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 msgr2=0x7f61e414bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61d00098c0 tx=0x7f61d00046c0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 shutdown_connections 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f61c406c940 0x7f61c406ee00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e415ee40 0x7f61e40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e40d57e0 0x7f61e414bf10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 --2- 172.21.15.43:0/166599707 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61e40d35d0 0x7f61e4161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 >> 172.21.15.43:0/166599707 conn(0x7f61e40d4cc0 msgr2=0x7f61e40b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 shutdown_connections 2024-04-26T20:28:00.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.570+0000 7f61eb000640 1 -- 172.21.15.43:0/166599707 wait complete. 2024-04-26T20:28:00.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:28:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.794+0000 7ff423200640 1 Processor -- start 2024-04-26T20:28:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.794+0000 7ff423200640 1 -- start start 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.795+0000 7ff423200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 0x7ff41c0d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.795+0000 7ff423200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff41c0d05b0 0x7ff41c0d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.795+0000 7ff423200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c0d0de0 con 0x7ff41c0d3db0 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.795+0000 7ff423200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff41c0d2ab0 con 0x7ff41c0d05b0 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.795+0000 7ff423200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c0d2d90 con 0x7ff41c15f180 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff421200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff41c0d3db0 0x7ff41c162d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38986/0 (socket says 172.21.15.43:38986) 2024-04-26T20:28:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff421200640 1 -- 172.21.15.43:0/3516378055 learned_addr learned my addr 172.21.15.43:0/3516378055 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:28:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930110268 0 0) 0x7ff41c0d0de0 con 0x7ff41c0d3db0 2024-04-26T20:28:00.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff41be00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 0x7ff41c0d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff41b400640 1 --2- 172.21.15.43:0/3516378055 >> v2:172.21.15.164:3300/0 conn(0x7ff41c0d05b0 0x7ff41c0d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.796+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003520 con 0x7ff41c0d3db0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 411821751 0 0) 0x7ff400003520 con 0x7ff41c0d3db0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 msgr2=0x7ff41c0d5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 0x7ff41c0d5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 >> v2:172.21.15.164:3300/0 conn(0x7ff41c0d05b0 msgr2=0x7ff41c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 --2- 172.21.15.43:0/3516378055 >> v2:172.21.15.164:3300/0 conn(0x7ff41c0d05b0 0x7ff41c0d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c0d3070 con 0x7ff41c0d3db0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7ff414001670 con 0x7ff41c0d3db0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 >> v1:172.21.15.43:6789/0 conn(0x7ff41c0d3db0 legacy=0x7ff41c162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 0x7ff4000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0d2ab0 con 0x7ff4000039b0 2024-04-26T20:28:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0d2d90 con 0x7ff400004800 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff40000ba70 con 0x7ff41c15f180 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.797+0000 7ff41be00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41be00640 1 -- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 msgr2=0x7ff4000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41be00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c15f180 0x7ff4000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41be00640 1 -- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff400004800 msgr2=0x7ff400004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41b400640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41be00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff400004800 0x7ff400004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.798+0000 7ff41be00640 1 -- 172.21.15.43:0/3516378055 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400003520 con 0x7ff4000039b0 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.799+0000 7ff41be00640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009fc0 tx=0x7ff400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf4df1556f3e077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.799+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c009c10 con 0x7ff4000039b0 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.799+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c036070 con 0x7ff4000039b0 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.799+0000 7ff41aa00640 1 -- 172.21.15.43:0/3516378055 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c02d070 con 0x7ff4000039b0 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 -- 172.21.15.43:0/3516378055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 msgr2=0x7ff400003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 --2- 172.21.15.43:0/3516378055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff400003dd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009fc0 tx=0x7ff400005830 comp rx=0 tx=0).stop 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff421200640 1 -- 172.21.15.43:0/3516378055 reap_dead start 2024-04-26T20:28:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 -- 172.21.15.43:0/3516378055 shutdown_connections 2024-04-26T20:28:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 -- 172.21.15.43:0/3516378055 >> 172.21.15.43:0/3516378055 conn(0x7ff41c0d4ee0 msgr2=0x7ff41c0d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 -- 172.21.15.43:0/3516378055 shutdown_connections 2024-04-26T20:28:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.800+0000 7ff423200640 1 -- 172.21.15.43:0/3516378055 wait complete. 2024-04-26T20:28:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.802+0000 7ff423200640 1 Processor -- start 2024-04-26T20:28:00.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.802+0000 7ff423200640 1 -- start start 2024-04-26T20:28:00.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c1f8460 0x7ff41c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff41c15bea0 0x7ff41c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0d09d0 con 0x7ff4000039b0 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c15f560 con 0x7ff41c15bea0 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff423200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c0d0b50 con 0x7ff41c1f8460 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff41be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36704/0 (socket says 172.21.15.43:36704) 2024-04-26T20:28:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.803+0000 7ff41be00640 1 -- 172.21.15.43:0/2020789213 learned_addr learned my addr 172.21.15.43:0/2020789213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff421200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff41c15bea0 0x7ff41c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41b400640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c1f8460 0x7ff41c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 -- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c1f8460 msgr2=0x7ff41c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c1f8460 0x7ff41c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 -- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff41c15bea0 msgr2=0x7ff41c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff41c15bea0 0x7ff41c15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:00.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 -- 172.21.15.43:0/2020789213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c15c860 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff41be00640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02af10 tx=0x7ff40c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c009c10 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c003bb0 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004020 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.804+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c1f8c40 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.805+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1f9200 con 0x7ff4000039b0 2024-04-26T20:28:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.806+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c075d00 con 0x7ff4000039b0 2024-04-26T20:28:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.807+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff40c005d10 con 0x7ff4000039b0 2024-04-26T20:28:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.808+0000 7ff413e00640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.808+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff40c0ad250 con 0x7ff4000039b0 2024-04-26T20:28:00.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.808+0000 7ff41b400640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:00.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.809+0000 7ff41b400640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 0x7ff3f806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff41c1f83c0 tx=0x7ff404008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:00.813+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff41c075d00 con 0x7ff4000039b0 2024-04-26T20:28:01.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.062+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff41c06c500 con 0x7ff4000039b0 2024-04-26T20:28:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.063+0000 7ff413e00640 1 -- 172.21.15.43:0/2020789213 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7ff40c07b4e0 con 0x7ff4000039b0 2024-04-26T20:28:01.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:28:01.065 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:28:01.065 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:28:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.066+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 msgr2=0x7ff3f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.066+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 0x7ff3f806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff41c1f83c0 tx=0x7ff404008040 comp rx=0 tx=0).stop 2024-04-26T20:28:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.066+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 msgr2=0x7ff41c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.066+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02af10 tx=0x7ff40c009750 comp rx=0 tx=0).stop 2024-04-26T20:28:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 shutdown_connections 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff3f806c8f0 0x7ff3f806edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff41c1f8460 0x7ff41c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff41c15bea0 0x7ff41c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 --2- 172.21.15.43:0/2020789213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4000039b0 0x7ff41c0cb5e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 >> 172.21.15.43:0/2020789213 conn(0x7ff41c0d4ee0 msgr2=0x7ff41c0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 shutdown_connections 2024-04-26T20:28:01.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.067+0000 7ff423200640 1 -- 172.21.15.43:0/2020789213 wait complete. 2024-04-26T20:28:01.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:27:49.342228+0000', 'modified': '2024-04-26T20:27:52.451048+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5520': {'gid': 5520, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6837/37476793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6835', 'nonce': 37476793}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 37476793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5520, 'qdb_cluster': [5520]} 2024-04-26T20:28:01.491 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:28:01.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:28:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.715+0000 7f45f2800640 1 Processor -- start 2024-04-26T20:28:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.715+0000 7f45f2800640 1 -- start start 2024-04-26T20:28:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f2800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f45ec0b5b80 0x7f45ec0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f2800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ec0d0480 con 0x7f45ec0d2800 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f2800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45ec0d0720 con 0x7f45ec0d1620 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f2800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f45ec0d0ce0 con 0x7f45ec0b5b80 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f1e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45ec0d2800 0x7f45ec0d2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38992/0 (socket says 172.21.15.43:38992) 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.717+0000 7f45f1e00640 1 -- 172.21.15.43:0/3641677412 learned_addr learned my addr 172.21.15.43:0/3641677412 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3641677412 >> v2:172.21.15.164:3300/0 conn(0x7f45ec0b5b80 0x7f45ec0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119636940 0 0) 0x7f45ec0d0480 con 0x7f45ec0d2800 2024-04-26T20:28:01.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d4003520 con 0x7f45ec0d2800 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1716054161 0 0) 0x7f45d4003520 con 0x7f45ec0d2800 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 >> v2:172.21.15.164:3300/0 conn(0x7f45ec0b5b80 msgr2=0x7f45ec0d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 --2- 172.21.15.43:0/3641677412 >> v2:172.21.15.164:3300/0 conn(0x7f45ec0b5b80 0x7f45ec0d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 >> v1:172.21.15.164:6789/0 conn(0x7f45ec0d1620 legacy=0x7f45ec0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec0d0a00 con 0x7f45ec0d2800 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f45e40019f0 con 0x7f45ec0d2800 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.718+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 >> v1:172.21.15.43:6789/0 conn(0x7f45ec0d2800 legacy=0x7f45ec0d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45d40039b0 0x7f45d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d4004800 0x7f45d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 0x7f45d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0d0ce0 con 0x7f45d40052e0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45d400c1b0 con 0x7f45d40039b0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f45d400c3b0 con 0x7f45d4004800 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 0x7f45d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1400640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45d40039b0 0x7f45d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d4004800 0x7f45d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 -- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d4004800 msgr2=0x7f45d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d4004800 0x7f45d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 -- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45d40039b0 msgr2=0x7f45d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45d40039b0 0x7f45d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1e00640 1 -- 172.21.15.43:0/3641677412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d4003520 con 0x7f45d40052e0 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d4004800 0x7f45d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.719+0000 7f45f1400640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45d40039b0 0x7f45d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.720+0000 7f45f1e00640 1 -- 172.21.15.43:0/3641677412 reap_dead start 2024-04-26T20:28:01.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.720+0000 7f45f1e00640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 0x7f45d400b9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f45e4002230 tx=0x7f45d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68e9e42403995c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400a4a0 con 0x7f45d40052e0 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e402f070 con 0x7f45d40052e0 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45ebe00640 1 -- 172.21.15.43:0/3641677412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e402bcb0 con 0x7f45d40052e0 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 -- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 msgr2=0x7f45d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 0x7f45d400b9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f45e4002230 tx=0x7f45d4005830 comp rx=0 tx=0).stop 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 -- 172.21.15.43:0/3641677412 shutdown_connections 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 --2- 172.21.15.43:0/3641677412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45d40052e0 0x7f45d400b9c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 -- 172.21.15.43:0/3641677412 >> 172.21.15.43:0/3641677412 conn(0x7f45ec0d4c30 msgr2=0x7f45ec0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.721+0000 7f45f2800640 1 -- 172.21.15.43:0/3641677412 shutdown_connections 2024-04-26T20:28:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.722+0000 7f45f2800640 1 -- 172.21.15.43:0/3641677412 wait complete. 2024-04-26T20:28:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.723+0000 7f45f2800640 1 Processor -- start 2024-04-26T20:28:01.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.723+0000 7f45f2800640 1 -- start start 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 0x7f45ec0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ec0ca130 0x7f45ec0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0d58d0 con 0x7f45ec0caad0 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0d5750 con 0x7f45ec0ca130 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.724+0000 7f45f2800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec0d5a50 con 0x7f45d40052e0 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36722/0 (socket says 172.21.15.43:36722) 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 -- 172.21.15.43:0/4156234508 learned_addr learned my addr 172.21.15.43:0/4156234508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1400640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 0x7f45ec0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 -- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 msgr2=0x7f45ec0b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 0x7f45ec0b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f0a00640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ec0ca130 0x7f45ec0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 -- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ec0ca130 msgr2=0x7f45ec0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ec0ca130 0x7f45ec0ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1e00640 1 -- 172.21.15.43:0/4156234508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec0b9ff0 con 0x7f45ec0caad0 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.725+0000 7f45f1400640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 0x7f45ec0b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f45e400a940 tx=0x7f45e402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45e9600640 1 -- 172.21.15.43:0/4156234508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e402bc60 con 0x7f45ec0caad0 2024-04-26T20:28:01.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45e9600640 1 -- 172.21.15.43:0/4156234508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e402be00 con 0x7f45ec0caad0 2024-04-26T20:28:01.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45e9600640 1 -- 172.21.15.43:0/4156234508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40346b0 con 0x7f45ec0caad0 2024-04-26T20:28:01.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec0ba270 con 0x7f45ec0caad0 2024-04-26T20:28:01.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.726+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec0caf70 con 0x7f45ec0caad0 2024-04-26T20:28:01.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.729+0000 7f45e9600640 1 -- 172.21.15.43:0/4156234508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45e4034850 con 0x7f45ec0caad0 2024-04-26T20:28:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.730+0000 7f45e9600640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 0x7f45cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.730+0000 7f45e9600640 1 -- 172.21.15.43:0/4156234508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f45e40abdd0 con 0x7f45ec0caad0 2024-04-26T20:28:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.730+0000 7f45f1400640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 0x7f45cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.730+0000 7f45f6d7f740 1 Processor -- start 2024-04-26T20:28:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.731+0000 7f45f6d7f740 1 -- start start 2024-04-26T20:28:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.731+0000 7f45f1400640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 0x7f45cc06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007fd0 tx=0x7f45e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.731+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56357e206600 con 0x56357e2dee80 2024-04-26T20:28:01.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56357e1a3100 con 0x56357e2a27b0 2024-04-26T20:28:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x56357e2025c0 con 0x56357e2cd270 2024-04-26T20:28:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36728/0 (socket says 172.21.15.43:36728) 2024-04-26T20:28:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 -- 172.21.15.43:0/74981597 learned_addr learned my addr 172.21.15.43:0/74981597 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f1e00640 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 -- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 msgr2=0x56357e2cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 -- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 msgr2=0x56357e2a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f0a00640 1 -- 172.21.15.43:0/74981597 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357df182d0 con 0x56357e2dee80 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.732+0000 7f45f1e00640 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.733+0000 7f45f0a00640 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d860 tx=0x7f45d800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73990408b74d846c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.733+0000 7f45ddc00640 1 -- 172.21.15.43:0/74981597 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004230 con 0x56357e2dee80 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.733+0000 7f45ddc00640 1 -- 172.21.15.43:0/74981597 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d80043d0 con 0x56357e2dee80 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.733+0000 7f45ddc00640 1 -- 172.21.15.43:0/74981597 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d800f960 con 0x56357e2dee80 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 -- 172.21.15.43:0/74981597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 msgr2=0x56357e2a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45d800d860 tx=0x7f45d800dd20 comp rx=0 tx=0).stop 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 -- 172.21.15.43:0/74981597 shutdown_connections 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2a2b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/74981597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 -- 172.21.15.43:0/74981597 >> 172.21.15.43:0/74981597 conn(0x56357e236d10 msgr2=0x56357e2e8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 -- 172.21.15.43:0/74981597 shutdown_connections 2024-04-26T20:28:01.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.734+0000 7f45f6d7f740 1 -- 172.21.15.43:0/74981597 wait complete. 2024-04-26T20:28:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.735+0000 7f45f6d7f740 1 -- start start 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.735+0000 7f45f6d7f740 1 Processor -- start 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56357e18df70 con 0x56357e2dee80 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56357e2cadc0 con 0x56357e2a27b0 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x56357e2e0a50 con 0x56357e2cd270 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f6d7f740 10 client.0 _ll_register_callbacks cb 0x56357e1fc480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36732/0 (socket says 172.21.15.43:36732) 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.736+0000 7f45f1e00640 1 -- 172.21.15.43:0/4159705793 learned_addr learned my addr 172.21.15.43:0/4159705793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1400640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f0a00640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 -- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 msgr2=0x56357e2cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 -- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 msgr2=0x56357e2bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2bca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 -- 172.21.15.43:0/4159705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357dfe26b0 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1400640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f1e00640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45e40aa100 tx=0x7f45e407a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c36e4f91f0aefc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45d2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40820b0 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f45e4082250 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.737+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d80043d0 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f45e4034b40 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 10 client.5709 fetch_fsmap learned FSMap version 18 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x56357e2e0790 con 0x56357e2dee80 2024-04-26T20:28:01.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d80043d0 con 0x56357e2dee80 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f45e4034ce0 con 0x56357e2dee80 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 10 client.5709 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 10 client.5709 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 20 client.5709 populate_metadata read hostname 'smithi043' 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2df710 0x56357e2dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 4 client.5709 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 --> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357e214210 con 0x56357e2df710 2024-04-26T20:28:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.738+0000 7f45f1400640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2df710 0x56357e2dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.739+0000 7f45f1400640 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2df710 0x56357e2dfaf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aec83110f715c3f server_cookie=512dbaf85de68f33 in_seq=0 out_seq=0 2024-04-26T20:28:01.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.739+0000 7f45d2000640 10 client.5709 ms_handle_connect on v2:172.21.15.43:6835/37476793 2024-04-26T20:28:01.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.740+0000 7f45d2000640 1 -- 172.21.15.43:0/4159705793 <== mds.0 v2:172.21.15.43:6835/37476793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357e214210 con 0x56357e2df710 2024-04-26T20:28:01.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.740+0000 7f45d2000640 10 client.5709 handle_command_reply: tid=0 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.740+0000 7f45f6d7f740 1 client.5709 shutdown 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.740+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 msgr2=0x56357e2e1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.740+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f45e40aa100 tx=0x7f45e407a1a0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 shutdown_connections 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2df710 0x56357e2dfaf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2cd270 0x56357e2cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2e1900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/4159705793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e2bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 >> 172.21.15.43:0/4159705793 conn(0x56357e236d10 msgr2=0x56357e2eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45d2000640 0 client.5709 ms_handle_reset on v2:172.21.15.43:6835/37476793 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 shutdown_connections 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 1 -- 172.21.15.43:0/4159705793 wait complete. 2024-04-26T20:28:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.741+0000 7f45f6d7f740 20 client.5709 trim_cache size 0 max 16384 2024-04-26T20:28:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.761+0000 7f45f6d7f740 1 Processor -- start 2024-04-26T20:28:01.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.761+0000 7f45f6d7f740 1 -- start start 2024-04-26T20:28:01.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e298df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2dee80 0x56357e2eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec2011a0 con 0x56357e2cd270 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec200500 con 0x56357e2a27b0 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec200b50 con 0x56357e2dee80 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36740/0 (socket says 172.21.15.43:36740) 2024-04-26T20:28:01.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.762+0000 7f45f0a00640 1 -- 172.21.15.43:0/3483715329 learned_addr learned my addr 172.21.15.43:0/3483715329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f1e00640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2dee80 0x56357e2eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f1400640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e298df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 -- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2dee80 msgr2=0x56357e2eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2dee80 0x56357e2eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 -- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 msgr2=0x56357e298df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e298df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 -- 172.21.15.43:0/3483715329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357e2a0b10 con 0x56357e2cd270 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f1400640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e298df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45f0a00640 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f45d8004000 tx=0x7f45d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0919d994dc0814d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.763+0000 7f45dd200640 1 -- 172.21.15.43:0/3483715329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004030 con 0x56357e2cd270 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45dd200640 1 -- 172.21.15.43:0/3483715329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d80041f0 con 0x56357e2cd270 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45dd200640 1 -- 172.21.15.43:0/3483715329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d800f760 con 0x56357e2cd270 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 -- 172.21.15.43:0/3483715329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 msgr2=0x56357e2dfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f45d8004000 tx=0x7f45d8014000 comp rx=0 tx=0).stop 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 -- 172.21.15.43:0/3483715329 shutdown_connections 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2dee80 0x56357e2eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2cd270 0x56357e2dfce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/3483715329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2a27b0 0x56357e298df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 -- 172.21.15.43:0/3483715329 >> 172.21.15.43:0/3483715329 conn(0x56357e236d10 msgr2=0x56357e2bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 -- 172.21.15.43:0/3483715329 shutdown_connections 2024-04-26T20:28:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.764+0000 7f45f6d7f740 1 -- 172.21.15.43:0/3483715329 wait complete. 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.765+0000 7f45f6d7f740 1 -- start start 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.765+0000 7f45f6d7f740 1 Processor -- start 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2a27b0 0x56357e36a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2cd270 0x56357e2eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56357e2e20c0 con 0x56357e2dee80 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56357e18df70 con 0x56357e2cd270 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x56357e2fb780 con 0x56357e2a27b0 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.766+0000 7f45f6d7f740 10 client.0 _ll_register_callbacks cb 0x7f45e402b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:28:01.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36746/0 (socket says 172.21.15.43:36746) 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 -- 172.21.15.43:0/509985298 learned_addr learned my addr 172.21.15.43:0/509985298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1400640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2a27b0 0x56357e36a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 -- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2a27b0 msgr2=0x56357e36a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2a27b0 0x56357e36a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 -- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2cd270 msgr2=0x56357e2eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2cd270 0x56357e2eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 -- 172.21.15.43:0/509985298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357e2fb170 con 0x56357e2dee80 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45f1e00640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f45e407a210 tx=0x7f45e40aa030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac72b9010dd7bb5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45d2000640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.767+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40792c0 con 0x56357e2dee80 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f45e40820b0 con 0x56357e2dee80 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d80041f0 con 0x56357e2dee80 2024-04-26T20:28:01.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f45e4075c00 con 0x56357e2dee80 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 10 client.5715 fetch_fsmap learned FSMap version 18 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x56357e2fb170 con 0x56357e2dee80 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d80041f0 con 0x56357e2dee80 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f45e40756b0 con 0x56357e2dee80 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 10 client.5715 fetch_fsmap finished waiting for FSMap version 18 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 10 client.5715 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 20 client.5715 populate_metadata read hostname 'smithi043' 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2cbc60 0x56357e30a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 4 client.5715 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.768+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 --> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] -- command(tid 0: {"prefix": "status"}) -- 0x56357e2d0fc0 con 0x56357e2cbc60 2024-04-26T20:28:01.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.769+0000 7f45f1400640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2cbc60 0x56357e30a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:01.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.769+0000 7f45f1400640 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2cbc60 0x56357e30a950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5e7bf0b8bff99e server_cookie=1fcc15922008961e in_seq=0 out_seq=0 2024-04-26T20:28:01.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.769+0000 7f45d2000640 10 client.5715 ms_handle_connect on v2:172.21.15.43:6835/37476793 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45d2000640 1 -- 172.21.15.43:0/509985298 <== mds.0 v2:172.21.15.43:6835/37476793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56357e2d0fc0 con 0x56357e2cbc60 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45d2000640 10 client.5715 handle_command_reply: tid=0 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 client.5715 shutdown 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 msgr2=0x56357e2ebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f45e407a210 tx=0x7f45e40aa030 comp rx=0 tx=0).stop 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 shutdown_connections 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:6835/37476793,v1:172.21.15.43:6837/37476793] conn(0x56357e2cbc60 0x56357e30a950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56357e2dee80 0x56357e2ebd30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56357e2cd270 0x56357e2eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 --2- 172.21.15.43:0/509985298 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x56357e2a27b0 0x56357e36a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45d2000640 0 client.5715 ms_handle_reset on v2:172.21.15.43:6835/37476793 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 >> 172.21.15.43:0/509985298 conn(0x56357e236d10 msgr2=0x56357e2bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 shutdown_connections 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 1 -- 172.21.15.43:0/509985298 wait complete. 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.770+0000 7f45f6d7f740 20 client.5715 trim_cache size 0 max 16384 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6a354559-7839-4522-9cd7-4f1548bb621f", 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5520, 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 11.887845092999999, 2024-04-26T20:28:01.775 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 18, 2024-04-26T20:28:01.776 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 38, 2024-04-26T20:28:01.776 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 38, 2024-04-26T20:28:01.776 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 20.814977852999998 2024-04-26T20:28:01.776 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 msgr2=0x7f45cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 0x7f45cc06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007fd0 tx=0x7f45e0009040 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 msgr2=0x7f45ec0cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f45e400a940 tx=0x7f45e402d070 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 shutdown_connections 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f45cc06c8f0 0x7f45cc06edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45ec0caad0 0x7f45ec0cd760 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45ec0ca130 0x7f45ec0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 --2- 172.21.15.43:0/4156234508 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45d40052e0 0x7f45ec0b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 >> 172.21.15.43:0/4156234508 conn(0x7f45ec0d4c30 msgr2=0x7f45ec0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 shutdown_connections 2024-04-26T20:28:01.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.771+0000 7f45f2800640 1 -- 172.21.15.43:0/4156234508 wait complete. 2024-04-26T20:28:01.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:28:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.996+0000 7f083b200640 1 Processor -- start 2024-04-26T20:28:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.996+0000 7f083b200640 1 -- start start 2024-04-26T20:28:01.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.998+0000 7f083b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.998+0000 7f083b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f083c160f60 0x7f083c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.998+0000 7f083b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083c0b50e0 con 0x7f083c161460 2024-04-26T20:28:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.998+0000 7f083b200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f083c0b4560 con 0x7f083c160f60 2024-04-26T20:28:02.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.998+0000 7f083b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0b4840 con 0x7f083c1463e0 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f083c160f60 0x7f083c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f083c160f60 0x7f083c06e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36754/0 (socket says 172.21.15.43:36754) 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 -- 172.21.15.43:0/2593921536 learned_addr learned my addr 172.21.15.43:0/2593921536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f0839e00640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 -- 172.21.15.43:0/2593921536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 msgr2=0x7f083c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 -- 172.21.15.43:0/2593921536 >> v1:172.21.15.164:6789/0 conn(0x7f083c161460 legacy=0x7f083c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:28:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:01.999+0000 7f083a800640 1 -- 172.21.15.43:0/2593921536 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0b4b20 con 0x7f083c160f60 2024-04-26T20:28:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.000+0000 7f083a800640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c06e050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0834009e50 tx=0x7f083402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f7522e728c489f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.001+0000 7f0838a00640 1 -- 172.21.15.43:0/2593921536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834036070 con 0x7f083c160f60 2024-04-26T20:28:02.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.001+0000 7f0838a00640 1 -- 172.21.15.43:0/2593921536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0834031600 con 0x7f083c160f60 2024-04-26T20:28:02.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.001+0000 7f0838a00640 1 -- 172.21.15.43:0/2593921536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08340319d0 con 0x7f083c160f60 2024-04-26T20:28:02.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.001+0000 7f083b200640 1 -- 172.21.15.43:0/2593921536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 msgr2=0x7f083c06e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.001+0000 7f083b200640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c06e050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0834009e50 tx=0x7f083402a640 comp rx=0 tx=0).stop 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 -- 172.21.15.43:0/2593921536 shutdown_connections 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c06e050 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 --2- 172.21.15.43:0/2593921536 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 -- 172.21.15.43:0/2593921536 >> 172.21.15.43:0/2593921536 conn(0x7f083c0b26b0 msgr2=0x7f083c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 -- 172.21.15.43:0/2593921536 shutdown_connections 2024-04-26T20:28:02.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.002+0000 7f083b200640 1 -- 172.21.15.43:0/2593921536 wait complete. 2024-04-26T20:28:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.003+0000 7f083b200640 1 Processor -- start 2024-04-26T20:28:02.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.004+0000 7f083b200640 1 -- start start 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.004+0000 7f083b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.004+0000 7f083b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.004+0000 7f083b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 0x7f083c073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f083b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f083c0b5df0 con 0x7f083c160f60 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f083b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1467c0 con 0x7f083c161460 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f083b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f083c06e590 con 0x7f083c1463e0 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36764/0 (socket says 172.21.15.43:36764) 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 -- 172.21.15.43:0/3631608563 learned_addr learned my addr 172.21.15.43:0/3631608563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839e00640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f083a800640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 0x7f083c073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 -- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 msgr2=0x7f083c0afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 -- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 msgr2=0x7f083c073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 0x7f083c073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f0839400640 1 -- 172.21.15.43:0/3631608563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0aeca0 con 0x7f083c160f60 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.005+0000 7f083a800640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 0x7f083c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f0839400640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f082800b3f0 tx=0x7f082800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828015020 con 0x7f083c160f60 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0828011830 con 0x7f083c160f60 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828011c20 con 0x7f083c160f60 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c0aef80 con 0x7f083c160f60 2024-04-26T20:28:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.006+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c0af540 con 0x7f083c160f60 2024-04-26T20:28:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.007+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c0b0280 con 0x7f083c160f60 2024-04-26T20:28:02.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.009+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0828018020 con 0x7f083c160f60 2024-04-26T20:28:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.009+0000 7f0832000640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 0x7f081c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:28:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.010+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f082808b850 con 0x7f083c160f60 2024-04-26T20:28:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.010+0000 7f0839e00640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 0x7f081c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:28:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.011+0000 7f0839e00640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 0x7f081c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:28:02.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.015+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f083c0b0280 con 0x7f083c160f60 2024-04-26T20:28:02.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.261+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f083c069350 con 0x7f083c160f60 2024-04-26T20:28:02.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.262+0000 7f0832000640 1 -- 172.21.15.43:0/3631608563 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f082801f030 con 0x7f083c160f60 2024-04-26T20:28:02.264 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 msgr2=0x7f081c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 0x7f081c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).stop 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 msgr2=0x7f083c13bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f082800b3f0 tx=0x7f082800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 shutdown_connections 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f081c06c8f0 0x7f081c06edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f083c161460 0x7f083c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f083c160f60 0x7f083c13bf10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 --2- 172.21.15.43:0/3631608563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f083c1463e0 0x7f083c0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:28:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 >> 172.21.15.43:0/3631608563 conn(0x7f083c0b26b0 msgr2=0x7f083c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:28:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.265+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 shutdown_connections 2024-04-26T20:28:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02.266+0000 7f083b200640 1 -- 172.21.15.43:0/3631608563 wait complete. 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:28:02.286 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:28:02.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:02.287 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-26T20:28:02.335 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:28:02.336 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42509sec preferred_lft 42509sec 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:28:02.337 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 22:c5:59:a2:ed:8d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::68c8:72ff:fee1:53b7/64 scope link 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 22:c5:59:a2:ed:8d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::20c5:59ff:fea2:ed8d/64 scope link 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 4e:40:2d:ac:f6:ac brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4c40:2dff:feac:f6ac/64 scope link 2024-04-26T20:28:02.338 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:28:02.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:02.339 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:28:02.413 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-26T20:28:02.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:02.417 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-26T20:28:02.469 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-26T20:28:02.470 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:28:02.470 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:28:02.470 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:28:02.470 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:28:02.471 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-26T20:28:02.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:28:02.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:28:02.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:02.700 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:28:02.752 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:28:02.752 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:28:02.753 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:28:02.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:02.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-26T20:28:02.838 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:28:02.838 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:28:02.838 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:02.896 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:28:02.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:28:02.896 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:02.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:28:03.046 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:28:03.104 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:28:03.105 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:28:03.106 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:28:03.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:03.107 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:28:03.159 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:28:03.159 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:28:03.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-26T20:28:03.234 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:28:03.277 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-26T20:28:03.278 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:28:03.279 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:28:03.280 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-26T20:28:03.280 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:28:03.280 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:28:03.280 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:28:03.280 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:28:03.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:03.281 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-26T20:28:03.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-26T20:28:03.347+0000 7fa07b80e480 -1 init, newargv = 0x563af023c140 newargc=15 2024-04-26T20:28:03.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48756]: starting ceph client 2024-04-26T20:28:03.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48756]: starting fuse 2024-04-26T20:28:03.369 INFO:teuthology.orchestra.run.smithi043.stdout:42 2024-04-26T20:28:03.369 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:28:03.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:28:03.369 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-26T20:28:03.369 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:28:03.370 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-26T20:28:03.371 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-26T20:28:03.559 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.48756.asok 2024-04-26T20:28:03.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:28:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:28:03.577 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48756.asok 2024-04-26T20:28:03.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:03.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48756.asok status 2024-04-26T20:28:03.813 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:28:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-26T20:28:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:28:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "48756", 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5727, 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-26T20:28:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 5727 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1554613822 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:28:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1554613822 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.5727 192.168.0.1:0/1554613822", 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1554613822", 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 18, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 38, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-26T20:28:03.816 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:28:03.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:03.829 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-26T20:28:03.829 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:03.884 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-26T20:28:03.884 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:03.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:03.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:28:03.976 DEBUG:teuthology.orchestra.run.smithi043:> mktemp -d --suffix=xfstests-dev 2024-04-26T20:28:04.033 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.dmSIwu2Lebxfstests-dev 2024-04-26T20:28:04.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:04.034 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.dmSIwu2Lebxfstests-dev) 2024-04-26T20:28:04.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:04.123 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.dmSIwu2Lebxfstests-dev) 2024-04-26T20:28:04.190 INFO:teuthology.orchestra.run.smithi043.stderr:Cloning into '/tmp/tmp.dmSIwu2Lebxfstests-dev'... 2024-04-26T20:28:05.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:05.272 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-26T20:28:05.345 DEBUG:teuthology.orchestra.run.smithi043:> mktemp -d --suffix=test 2024-04-26T20:28:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.10sH9QpXIytest 2024-04-26T20:28:05.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:28:05.401 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-26T20:28:05.459 DEBUG:teuthology.orchestra.run.smithi043:> mktemp -d --suffix=scratch 2024-04-26T20:28:05.511 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.AeX8EKgNejscratch 2024-04-26T20:28:05.512 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-26T20:28:05.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-26T20:28:06.328 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:05:21 ago on Fri 26 Apr 2024 08:22:45 PM UTC. 2024-04-26T20:28:06.420 INFO:teuthology.orchestra.run.smithi043.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:28:06.421 INFO:teuthology.orchestra.run.smithi043.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:28:06.421 INFO:teuthology.orchestra.run.smithi043.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-26T20:28:06.422 INFO:teuthology.orchestra.run.smithi043.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-26T20:28:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-26T20:28:06.425 INFO:teuthology.orchestra.run.smithi043.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-04-26T20:28:06.426 INFO:teuthology.orchestra.run.smithi043.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-26T20:28:06.427 INFO:teuthology.orchestra.run.smithi043.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-04-26T20:28:06.429 INFO:teuthology.orchestra.run.smithi043.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-04-26T20:28:06.431 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:28:06.432 INFO:teuthology.orchestra.run.smithi043.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-26T20:28:06.434 INFO:teuthology.orchestra.run.smithi043.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-26T20:28:06.435 INFO:teuthology.orchestra.run.smithi043.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:28:06.435 INFO:teuthology.orchestra.run.smithi043.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-04-26T20:28:06.436 INFO:teuthology.orchestra.run.smithi043.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-04-26T20:28:06.437 INFO:teuthology.orchestra.run.smithi043.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-26T20:28:06.438 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-04-26T20:28:06.439 INFO:teuthology.orchestra.run.smithi043.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-26T20:28:06.440 INFO:teuthology.orchestra.run.smithi043.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-26T20:28:06.442 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:28:06.491 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-26T20:28:06.493 INFO:teuthology.orchestra.run.smithi043.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-devel x86_64 5.14.0-437.el9 appstream 16 M 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-04-26T20:28:06.494 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout:Install 17 Packages 2024-04-26T20:28:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:28:06.496 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 24 M 2024-04-26T20:28:06.496 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 78 M 2024-04-26T20:28:06.496 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-26T20:28:07.589 INFO:teuthology.orchestra.run.smithi043.stdout:(1/17): rmt-1.6-6.el9.x86_64.rpm 276 kB/s | 46 kB 00:00 2024-04-26T20:28:07.687 INFO:teuthology.orchestra.run.smithi043.stdout:(2/17): fio-engine-http-3.35-1.el9.x86_64.rpm 68 kB/s | 18 kB 00:00 2024-04-26T20:28:07.754 INFO:teuthology.orchestra.run.smithi043.stdout:(3/17): fio-engine-libaio-3.35-1.el9.x86_64.rpm 96 kB/s | 16 kB 00:00 2024-04-26T20:28:07.780 INFO:teuthology.orchestra.run.smithi043.stdout:(4/17): fio-engine-nbd-3.35-1.el9.x86_64.rpm 164 kB/s | 15 kB 00:00 2024-04-26T20:28:07.838 INFO:teuthology.orchestra.run.smithi043.stdout:(5/17): fio-engine-rados-3.35-1.el9.x86_64.rpm 190 kB/s | 16 kB 00:00 2024-04-26T20:28:07.872 INFO:teuthology.orchestra.run.smithi043.stdout:(6/17): fio-engine-rbd-3.35-1.el9.x86_64.rpm 180 kB/s | 16 kB 00:00 2024-04-26T20:28:07.955 INFO:teuthology.orchestra.run.smithi043.stdout:(7/17): fio-engine-rdma-3.35-1.el9.x86_64.rpm 178 kB/s | 21 kB 00:00 2024-04-26T20:28:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:(8/17): libcap-devel-2.48-9.el9.x86_64.rpm 446 kB/s | 41 kB 00:00 2024-04-26T20:28:08.148 INFO:teuthology.orchestra.run.smithi043.stdout:(9/17): fio-3.35-1.el9.x86_64.rpm 8.6 MB/s | 6.2 MB 00:00 2024-04-26T20:28:08.206 INFO:teuthology.orchestra.run.smithi043.stdout:(10/17): libnbd-1.18.1-3.el9.x86_64.rpm 1.0 MB/s | 166 kB 00:00 2024-04-26T20:28:08.340 INFO:teuthology.orchestra.run.smithi043.stdout:(11/17): udftools-2.2-5.el9.x86_64.rpm 1.1 MB/s | 149 kB 00:00 2024-04-26T20:28:08.623 INFO:teuthology.orchestra.run.smithi043.stdout:(12/17): sqlite-3.34.1-7.el9.x86_64.rpm 1.5 MB/s | 748 kB 00:00 2024-04-26T20:28:08.841 INFO:teuthology.orchestra.run.smithi043.stdout:(13/17): kernel-devel-5.14.0-437.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-26T20:28:08.866 INFO:teuthology.orchestra.run.smithi043.stdout:(14/17): gdbm-1.19-4.el9.x86_64.rpm 252 kB/s | 132 kB 00:00 2024-04-26T20:28:09.116 INFO:teuthology.orchestra.run.smithi043.stdout:(15/17): gdbm-devel-1.19-4.el9.x86_64.rpm 119 kB/s | 59 kB 00:00 2024-04-26T20:28:09.308 INFO:teuthology.orchestra.run.smithi043.stdout:(16/17): dump-0.4-0.51.b47.el9.x86_64.rpm 189 kB/s | 88 kB 00:00 2024-04-26T20:28:09.400 INFO:teuthology.orchestra.run.smithi043.stdout:(17/17): indent-2.2.13-5.el9.x86_64.rpm 337 kB/s | 179 kB 00:00 2024-04-26T20:28:09.406 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:28:09.407 INFO:teuthology.orchestra.run.smithi043.stdout:Total 8.1 MB/s | 24 MB 00:02 2024-04-26T20:28:10.145 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-26T20:28:10.260 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-26T20:28:10.261 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-26T20:28:11.071 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-26T20:28:11.073 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-26T20:28:12.230 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-26T20:28:12.323 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 1/17 2024-04-26T20:28:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 2/17 2024-04-26T20:28:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-26T20:28:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-26T20:28:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-26T20:28:12.492 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-26T20:28:12.526 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-26T20:28:12.799 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-26T20:28:12.854 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fio-3.35-1.el9.x86_64 9/17 2024-04-26T20:28:12.898 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rmt-2:1.6-6.el9.x86_64 10/17 2024-04-26T20:28:12.958 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 11/17 2024-04-26T20:28:13.057 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 12/17 2024-04-26T20:28:13.129 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : indent-2.2.13-5.el9.x86_64 13/17 2024-04-26T20:28:13.194 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : udftools-2.2-5.el9.x86_64 14/17 2024-04-26T20:28:13.279 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 15/17 2024-04-26T20:28:25.507 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 16/17 2024-04-26T20:28:25.879 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-26T20:28:27.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:28:27.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:28:57.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:28:57.770 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:29:20.048 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-3.35-1.el9.x86_64 2/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-devel-5.14.0-437.el9.x86_64 9/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 11/17 2024-04-26T20:29:20.049 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/17 2024-04-26T20:29:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/17 2024-04-26T20:29:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/17 2024-04-26T20:29:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/17 2024-04-26T20:29:20.050 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/17 2024-04-26T20:29:20.695 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/17 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-437.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: udftools-2.2-5.el9.x86_64 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:29:20.696 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-26T20:29:21.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo useradd -m fsgqa 2024-04-26T20:29:21.289 INFO:teuthology.orchestra.run.smithi043.stderr:useradd: user 'fsgqa' already exists 2024-04-26T20:29:21.292 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-26T20:29:21.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo groupadd fsgqa 2024-04-26T20:29:21.356 INFO:teuthology.orchestra.run.smithi043.stderr:groupadd: group 'fsgqa' already exists 2024-04-26T20:29:21.359 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-26T20:29:21.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo useradd fsgqa2 2024-04-26T20:29:22.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo useradd 123456-fsgqa 2024-04-26T20:29:22.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-26T20:29:22.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.907+0000 7f25a6400640 1 Processor -- start 2024-04-26T20:29:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.907+0000 7f25a6400640 1 -- start start 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f25a6400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25a013fc90 0x7f25a0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f25a6400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a00d0da0 0x7f25a00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f25a6400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a015e3e0 con 0x7f25a015df70 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f25a6400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f25a00d1750 con 0x7f25a013fc90 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f25a6400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f25a0161a10 con 0x7f25a00d0da0 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f259f400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25a013fc90 0x7f25a0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f259f400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25a013fc90 0x7f25a0160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59028/0 (socket says 172.21.15.43:59028) 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f259f400640 1 -- 172.21.15.43:0/1899820067 learned_addr learned my addr 172.21.15.43:0/1899820067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:29:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.909+0000 7f259ea00640 1 --1- 172.21.15.43:0/1899820067 >> v1:172.21.15.43:6789/0 conn(0x7f25a015df70 0x7f25a0142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36852/0 (socket says 172.21.15.43:36852) 2024-04-26T20:29:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.910+0000 7f259fe00640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a00d0da0 0x7f25a00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.910+0000 7f259f400640 1 -- 172.21.15.43:0/1899820067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a00d0da0 msgr2=0x7f25a00d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.910+0000 7f259f400640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a00d0da0 0x7f25a00d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.910+0000 7f259f400640 1 -- 172.21.15.43:0/1899820067 >> v1:172.21.15.43:6789/0 conn(0x7f25a015df70 legacy=0x7f25a0142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.910+0000 7f259f400640 1 -- 172.21.15.43:0/1899820067 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a0161170 con 0x7f25a013fc90 2024-04-26T20:29:22.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.911+0000 7f259f400640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a013fc90 0x7f25a0160ae0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2590009870 tx=0x7f259002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ef9ccade82b10d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:22.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f259e000640 1 -- 172.21.15.43:0/1899820067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590037070 con 0x7f25a013fc90 2024-04-26T20:29:22.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f259e000640 1 -- 172.21.15.43:0/1899820067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2590032440 con 0x7f25a013fc90 2024-04-26T20:29:22.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f259e000640 1 -- 172.21.15.43:0/1899820067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590032810 con 0x7f25a013fc90 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 -- 172.21.15.43:0/1899820067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a013fc90 msgr2=0x7f25a0160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a013fc90 0x7f25a0160ae0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2590009870 tx=0x7f259002a250 comp rx=0 tx=0).stop 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 -- 172.21.15.43:0/1899820067 shutdown_connections 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a00d0da0 0x7f25a00d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 --2- 172.21.15.43:0/1899820067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a013fc90 0x7f25a0160ae0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.912+0000 7f25a6400640 1 -- 172.21.15.43:0/1899820067 >> 172.21.15.43:0/1899820067 conn(0x7f25a00682a0 msgr2=0x7f25a00686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.913+0000 7f25a6400640 1 -- 172.21.15.43:0/1899820067 shutdown_connections 2024-04-26T20:29:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.913+0000 7f25a6400640 1 -- 172.21.15.43:0/1899820067 wait complete. 2024-04-26T20:29:22.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.914+0000 7f25a6400640 1 Processor -- start 2024-04-26T20:29:22.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.915+0000 7f25a6400640 1 -- start start 2024-04-26T20:29:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.915+0000 7f25a6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f25a6400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a013fc90 0x7f25a013e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f25a6400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a015df70 0x7f25a015f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f25a6400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0142770 con 0x7f25a00d0da0 2024-04-26T20:29:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f25a6400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f25a0161730 con 0x7f25a015df70 2024-04-26T20:29:22.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f25a6400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f25a01433a0 con 0x7f25a013fc90 2024-04-26T20:29:22.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59030/0 (socket says 172.21.15.43:59030) 2024-04-26T20:29:22.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 -- 172.21.15.43:0/2801376703 learned_addr learned my addr 172.21.15.43:0/2801376703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 -- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a013fc90 msgr2=0x7f25a013e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a013fc90 0x7f25a013e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 -- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a015df70 msgr2=0x7f25a015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a015df70 0x7f25a015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.916+0000 7f259f400640 1 -- 172.21.15.43:0/2801376703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a0154ad0 con 0x7f25a00d0da0 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f259f400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f25900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590041020 con 0x7f25a00d0da0 2024-04-26T20:29:22.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25900306c0 con 0x7f25a00d0da0 2024-04-26T20:29:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a0154db0 con 0x7f25a00d0da0 2024-04-26T20:29:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590037040 con 0x7f25a00d0da0 2024-04-26T20:29:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.917+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a0155370 con 0x7f25a00d0da0 2024-04-26T20:29:22.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.919+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a0069350 con 0x7f25a00d0da0 2024-04-26T20:29:22.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.920+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2590044020 con 0x7f25a00d0da0 2024-04-26T20:29:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.920+0000 7f2597400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 0x7f258006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.921+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f25900ac4f0 con 0x7f25a00d0da0 2024-04-26T20:29:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.921+0000 7f259ea00640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 0x7f258006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:22.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.922+0000 7f259ea00640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 0x7f258006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:22.926+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f259004b030 con 0x7f25a00d0da0 2024-04-26T20:29:23.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.213+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f25a015ef60 con 0x7f25a00d0da0 2024-04-26T20:29:23.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.214+0000 7f2597400640 1 -- 172.21.15.43:0/2801376703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f259007a760 con 0x7f25a00d0da0 2024-04-26T20:29:23.215 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:29:23.216 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","modified":"2024-04-26T20:24:58.607719Z","created":"2024-04-26T20:24:58.607719Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3301","nonce":0},{"type":"v1","addr":"172.21.15.164:6790","nonce":0}]},"addr":"172.21.15.164:6790/0","public_addr":"172.21.15.164:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-26T20:29:23.216 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 msgr2=0x7f258006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 0x7f258006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c007a00 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 msgr2=0x7f25a0155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2590000c00 tx=0x7f25900046c0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 shutdown_connections 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f258006c8f0 0x7f258006edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25a013fc90 0x7f25a013e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25a015df70 0x7f25a015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 --2- 172.21.15.43:0/2801376703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a00d0da0 0x7f25a0155eb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 >> 172.21.15.43:0/2801376703 conn(0x7f25a00682a0 msgr2=0x7f25a013f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:23.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 shutdown_connections 2024-04-26T20:29:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.217+0000 7f25a6400640 1 -- 172.21.15.43:0/2801376703 wait complete. 2024-04-26T20:29:23.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-26T20:29:23.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.408+0000 7f370aa00640 1 Processor -- start 2024-04-26T20:29:23.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.408+0000 7f370aa00640 1 -- start start 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.410+0000 7f370aa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37041467c0 0x7f3704072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.410+0000 7f370aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37040b3ce0 0x7f37040d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.410+0000 7f370aa00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3704146ba0 con 0x7f37040b4140 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.410+0000 7f370aa00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3704073dd0 con 0x7f37041467c0 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.410+0000 7f370aa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f37040731c0 con 0x7f37040b3ce0 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37041467c0 0x7f3704072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3708a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f37040b4140 0x7f3704160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36854/0 (socket says 172.21.15.43:36854) 2024-04-26T20:29:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3708a00640 1 -- 172.21.15.43:0/2377552364 learned_addr learned my addr 172.21.15.43:0/2377552364 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:29:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703400640 1 --2- 172.21.15.43:0/2377552364 >> v2:172.21.15.164:3300/0 conn(0x7f37040b3ce0 0x7f37040d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703e00640 1 -- 172.21.15.43:0/2377552364 >> v2:172.21.15.164:3300/0 conn(0x7f37040b3ce0 msgr2=0x7f37040d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:29:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703e00640 1 --2- 172.21.15.43:0/2377552364 >> v2:172.21.15.164:3300/0 conn(0x7f37040b3ce0 0x7f37040d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703e00640 1 -- 172.21.15.43:0/2377552364 >> v1:172.21.15.43:6789/0 conn(0x7f37040b4140 legacy=0x7f3704160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3702a00640 1 -- 172.21.15.43:0/2377552364 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833391264 0 0) 0x7f3704146ba0 con 0x7f37040b4140 2024-04-26T20:29:23.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.411+0000 7f3703e00640 1 -- 172.21.15.43:0/2377552364 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37040734a0 con 0x7f37041467c0 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.412+0000 7f3703e00640 1 --2- 172.21.15.43:0/2377552364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041467c0 0x7f3704072b70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3704072320 tx=0x7f36f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b7c2b837c57e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f3702a00640 1 -- 172.21.15.43:0/2377552364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4037070 con 0x7f37041467c0 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f3702a00640 1 -- 172.21.15.43:0/2377552364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f4032440 con 0x7f37041467c0 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f3702a00640 1 -- 172.21.15.43:0/2377552364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4032810 con 0x7f37041467c0 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 -- 172.21.15.43:0/2377552364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041467c0 msgr2=0x7f3704072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 --2- 172.21.15.43:0/2377552364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041467c0 0x7f3704072b70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3704072320 tx=0x7f36f402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 -- 172.21.15.43:0/2377552364 shutdown_connections 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 --2- 172.21.15.43:0/2377552364 >> v2:172.21.15.164:3300/0 conn(0x7f37040b3ce0 0x7f37040d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 --2- 172.21.15.43:0/2377552364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041467c0 0x7f3704072b70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 -- 172.21.15.43:0/2377552364 >> 172.21.15.43:0/2377552364 conn(0x7f37040b2900 msgr2=0x7f37040b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.413+0000 7f370aa00640 1 -- 172.21.15.43:0/2377552364 shutdown_connections 2024-04-26T20:29:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.414+0000 7f370aa00640 1 -- 172.21.15.43:0/2377552364 wait complete. 2024-04-26T20:29:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.416+0000 7f370aa00640 1 Processor -- start 2024-04-26T20:29:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.416+0000 7f370aa00640 1 -- start start 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37040b3ce0 0x7f3704141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 0x7f370415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f370406f000 con 0x7f37041600e0 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3704073a60 con 0x7f37041467c0 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f370aa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f370406f180 con 0x7f37040b3ce0 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f3708a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f3708a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59052/0 (socket says 172.21.15.43:59052) 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.417+0000 7f3708a00640 1 -- 172.21.15.43:0/3750678253 learned_addr learned my addr 172.21.15.43:0/3750678253 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:29:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3703400640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 0x7f370415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 -- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37040b3ce0 msgr2=0x7f3704141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37040b3ce0 0x7f3704141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 -- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 msgr2=0x7f370415fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 0x7f370415fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 -- 172.21.15.43:0/3750678253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041412b0 con 0x7f37041600e0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3703400640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 0x7f370415fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f3708a00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c780 tx=0x7f36fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc018070 con 0x7f37041600e0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc00a5e0 con 0x7f37041600e0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00aa50 con 0x7f37041600e0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.418+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041404f0 con 0x7f37041600e0 2024-04-26T20:29:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.419+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3704140ab0 con 0x7f37041600e0 2024-04-26T20:29:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.420+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3704069350 con 0x7f37041600e0 2024-04-26T20:29:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.421+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36fc00abf0 con 0x7f37041600e0 2024-04-26T20:29:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.422+0000 7f36fbe00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 0x7f36e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:29:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.422+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f36fc08b2a0 con 0x7f37041600e0 2024-04-26T20:29:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.422+0000 7f3703e00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 0x7f36e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:29:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.423+0000 7f3703e00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 0x7f36e406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36f4034000 tx=0x7f36f402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:29:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.427+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36fc059510 con 0x7f37041600e0 2024-04-26T20:29:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.703+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f37040b48b0 con 0x7f37041600e0 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.703+0000 7f36fbe00640 1 -- 172.21.15.43:0/3750678253 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v11) ==== 82+0+151 (secure 0 0 0) 0x7f36fc058e10 con 0x7f37041600e0 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stdout:[client.admin] 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCaDSxm4fYrOhAAqANw7ERrPLoJQ7jiXby8OQ== 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow *" 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: caps mgr = "allow *" 2024-04-26T20:29:23.705 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow *" 2024-04-26T20:29:23.706 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow *" 2024-04-26T20:29:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 msgr2=0x7f36e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 0x7f36e406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36f4034000 tx=0x7f36f402c000 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 msgr2=0x7f37041519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c780 tx=0x7f36fc00cc40 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 shutdown_connections 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f36e406c8f0 0x7f36e406edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37041600e0 0x7f37041519a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37041467c0 0x7f370415fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 --2- 172.21.15.43:0/3750678253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37040b3ce0 0x7f3704141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 >> 172.21.15.43:0/3750678253 conn(0x7f37040b2900 msgr2=0x7f37040af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 shutdown_connections 2024-04-26T20:29:23.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:29:23.706+0000 7f370aa00640 1 -- 172.21.15.43:0/3750678253 wait complete. 2024-04-26T20:29:23.724 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-04-26T20:29:23.762 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.LfpifpZR6e 2024-04-26T20:29:23.762 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:29:23.762 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/tmp/tmp.LfpifpZR6e 2024-04-26T20:29:23.843 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:29:23.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/tmp/tmp.dmSIwu2Lebxfstests-dev/local.config 2024-04-26T20:29:23.942 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.10sH9QpXIytest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.AeX8EKgNejscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.43:6789 -k /tmp/tmp.LfpifpZR6e --client_mountpoint /test" 2024-04-26T20:29:23.942 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:29:23.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/tmp/tmp.dmSIwu2Lebxfstests-dev/ceph.exclude 2024-04-26T20:29:24.046 DEBUG:teuthology.orchestra.run.smithi043:> (cd /tmp/tmp.dmSIwu2Lebxfstests-dev && exec sudo make) 2024-04-26T20:29:24.163 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize -cfi 2024-04-26T20:29:24.719 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-26T20:29:24.719 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file './config.guess' 2024-04-26T20:29:24.730 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file './config.sub' 2024-04-26T20:29:24.740 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file './install-sh' 2024-04-26T20:29:24.752 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file './ltmain.sh' 2024-04-26T20:29:24.817 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-26T20:29:24.817 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-26T20:29:24.831 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-26T20:29:24.844 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-26T20:29:24.857 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-26T20:29:24.872 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-26T20:29:24.894 INFO:teuthology.orchestra.run.smithi043.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-26T20:29:24.899 INFO:teuthology.orchestra.run.smithi043.stdout:cp include/install-sh . 2024-04-26T20:29:24.902 INFO:teuthology.orchestra.run.smithi043.stdout:aclocal -I m4 2024-04-26T20:29:26.852 INFO:teuthology.orchestra.run.smithi043.stdout:autoheader 2024-04-26T20:29:27.302 INFO:teuthology.orchestra.run.smithi043.stdout:autoconf 2024-04-26T20:29:27.582 INFO:teuthology.orchestra.run.smithi043.stdout:./configure \ 2024-04-26T20:29:27.583 INFO:teuthology.orchestra.run.smithi043.stdout: --libexecdir=/usr/lib \ 2024-04-26T20:29:27.583 INFO:teuthology.orchestra.run.smithi043.stdout: --exec_prefix=/var/lib 2024-04-26T20:29:27.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:29:27.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:29:28.279 INFO:teuthology.orchestra.run.smithi043.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-26T20:29:28.279 INFO:teuthology.orchestra.run.smithi043.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-26T20:29:28.284 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to print strings... printf 2024-04-26T20:29:28.285 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gcc... gcc 2024-04-26T20:29:28.531 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether the C compiler works... yes 2024-04-26T20:29:28.531 INFO:teuthology.orchestra.run.smithi043.stdout:checking for C compiler default output file name... a.out 2024-04-26T20:29:28.602 INFO:teuthology.orchestra.run.smithi043.stdout:checking for suffix of executables... 2024-04-26T20:29:28.730 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether we are cross compiling... no 2024-04-26T20:29:28.782 INFO:teuthology.orchestra.run.smithi043.stdout:checking for suffix of object files... o 2024-04-26T20:29:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T20:29:28.872 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether gcc accepts -g... yes 2024-04-26T20:29:28.957 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T20:29:28.967 INFO:teuthology.orchestra.run.smithi043.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-26T20:29:28.975 INFO:teuthology.orchestra.run.smithi043.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-26T20:29:28.978 INFO:teuthology.orchestra.run.smithi043.stdout:checking for egrep... /bin/grep -E 2024-04-26T20:29:28.981 INFO:teuthology.orchestra.run.smithi043.stdout:checking for fgrep... /bin/grep -F 2024-04-26T20:29:28.992 INFO:teuthology.orchestra.run.smithi043.stdout:checking for ld used by gcc... /bin/ld 2024-04-26T20:29:28.998 INFO:teuthology.orchestra.run.smithi043.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-26T20:29:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-26T20:29:29.899 INFO:teuthology.orchestra.run.smithi043.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-26T20:29:29.900 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether ln -s works... yes 2024-04-26T20:29:29.926 INFO:teuthology.orchestra.run.smithi043.stdout:checking the maximum length of command line arguments... 1572864 2024-04-26T20:29:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-26T20:29:29.934 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-26T20:29:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:checking for /bin/ld option to reload object files... -r 2024-04-26T20:29:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:checking for objdump... objdump 2024-04-26T20:29:29.935 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to recognize dependent libraries... pass_all 2024-04-26T20:29:29.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dlltool... no 2024-04-26T20:29:29.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-26T20:29:29.937 INFO:teuthology.orchestra.run.smithi043.stdout:checking for ar... ar 2024-04-26T20:29:30.032 INFO:teuthology.orchestra.run.smithi043.stdout:checking for archiver @FILE support... @ 2024-04-26T20:29:30.032 INFO:teuthology.orchestra.run.smithi043.stdout:checking for strip... strip 2024-04-26T20:29:30.032 INFO:teuthology.orchestra.run.smithi043.stdout:checking for ranlib... ranlib 2024-04-26T20:29:30.033 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gawk... gawk 2024-04-26T20:29:30.196 INFO:teuthology.orchestra.run.smithi043.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-26T20:29:30.199 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sysroot... no 2024-04-26T20:29:30.210 INFO:teuthology.orchestra.run.smithi043.stdout:checking for a working dd... /bin/dd 2024-04-26T20:29:30.223 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-26T20:29:30.263 INFO:teuthology.orchestra.run.smithi043.stdout:checking for mt... no 2024-04-26T20:29:30.272 INFO:teuthology.orchestra.run.smithi043.stdout:checking if : is a manifest tool... no 2024-04-26T20:29:30.387 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T20:29:30.766 INFO:teuthology.orchestra.run.smithi043.stdout:checking for ANSI C header files... yes 2024-04-26T20:29:30.825 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/types.h... yes 2024-04-26T20:29:30.901 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/stat.h... yes 2024-04-26T20:29:30.971 INFO:teuthology.orchestra.run.smithi043.stdout:checking for stdlib.h... yes 2024-04-26T20:29:31.048 INFO:teuthology.orchestra.run.smithi043.stdout:checking for string.h... yes 2024-04-26T20:29:31.122 INFO:teuthology.orchestra.run.smithi043.stdout:checking for memory.h... yes 2024-04-26T20:29:31.194 INFO:teuthology.orchestra.run.smithi043.stdout:checking for strings.h... yes 2024-04-26T20:29:31.278 INFO:teuthology.orchestra.run.smithi043.stdout:checking for inttypes.h... yes 2024-04-26T20:29:31.355 INFO:teuthology.orchestra.run.smithi043.stdout:checking for stdint.h... yes 2024-04-26T20:29:31.436 INFO:teuthology.orchestra.run.smithi043.stdout:checking for unistd.h... yes 2024-04-26T20:29:31.511 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dlfcn.h... yes 2024-04-26T20:29:31.524 INFO:teuthology.orchestra.run.smithi043.stdout:checking for objdir... .libs 2024-04-26T20:29:31.724 INFO:teuthology.orchestra.run.smithi043.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-26T20:29:31.724 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-26T20:29:31.785 INFO:teuthology.orchestra.run.smithi043.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-26T20:29:31.852 INFO:teuthology.orchestra.run.smithi043.stdout:checking if gcc static flag -static works... no 2024-04-26T20:29:31.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking if gcc supports -c -o file.o... yes 2024-04-26T20:29:31.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-26T20:29:31.961 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-26T20:29:32.042 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether -lc should be explicitly linked in... no 2024-04-26T20:29:32.199 INFO:teuthology.orchestra.run.smithi043.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-26T20:29:32.199 INFO:teuthology.orchestra.run.smithi043.stdout:checking how to hardcode library paths into programs... immediate 2024-04-26T20:29:32.204 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether stripping libraries is possible... yes 2024-04-26T20:29:32.205 INFO:teuthology.orchestra.run.smithi043.stdout:checking if libtool supports shared libraries... yes 2024-04-26T20:29:32.205 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether to build shared libraries... yes 2024-04-26T20:29:32.205 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether to build static libraries... yes 2024-04-26T20:29:32.228 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gcc... (cached) gcc 2024-04-26T20:29:32.291 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T20:29:32.292 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T20:29:32.295 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T20:29:32.296 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gmake... /usr/bin/gmake 2024-04-26T20:29:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-26T20:29:32.297 INFO:teuthology.orchestra.run.smithi043.stdout:checking for tar... /bin/tar 2024-04-26T20:29:32.298 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gzip... /bin/gzip 2024-04-26T20:29:32.342 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether gcc -MM is supported... yes 2024-04-26T20:29:32.343 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sort... /bin/sort 2024-04-26T20:29:32.344 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether ln -s works... yes 2024-04-26T20:29:32.345 INFO:teuthology.orchestra.run.smithi043.stdout:checking for rpm... /bin/rpm 2024-04-26T20:29:32.358 INFO:teuthology.orchestra.run.smithi043.stdout:checking for rpmbuild... no 2024-04-26T20:29:32.440 INFO:teuthology.orchestra.run.smithi043.stdout:checking assert.h usability... yes 2024-04-26T20:29:32.469 INFO:teuthology.orchestra.run.smithi043.stdout:checking assert.h presence... yes 2024-04-26T20:29:32.469 INFO:teuthology.orchestra.run.smithi043.stdout:checking for assert.h... yes 2024-04-26T20:29:32.547 INFO:teuthology.orchestra.run.smithi043.stdout:checking bstring.h usability... no 2024-04-26T20:29:32.589 INFO:teuthology.orchestra.run.smithi043.stdout:checking bstring.h presence... no 2024-04-26T20:29:32.589 INFO:teuthology.orchestra.run.smithi043.stdout:checking for bstring.h... no 2024-04-26T20:29:32.657 INFO:teuthology.orchestra.run.smithi043.stdout:checking libgen.h usability... yes 2024-04-26T20:29:32.687 INFO:teuthology.orchestra.run.smithi043.stdout:checking libgen.h presence... yes 2024-04-26T20:29:32.687 INFO:teuthology.orchestra.run.smithi043.stdout:checking for libgen.h... yes 2024-04-26T20:29:32.775 INFO:teuthology.orchestra.run.smithi043.stdout:checking dirent.h usability... yes 2024-04-26T20:29:32.805 INFO:teuthology.orchestra.run.smithi043.stdout:checking dirent.h presence... yes 2024-04-26T20:29:32.805 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dirent.h... yes 2024-04-26T20:29:32.922 INFO:teuthology.orchestra.run.smithi043.stdout:checking errno.h usability... yes 2024-04-26T20:29:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:checking errno.h presence... yes 2024-04-26T20:29:32.952 INFO:teuthology.orchestra.run.smithi043.stdout:checking for errno.h... yes 2024-04-26T20:29:33.034 INFO:teuthology.orchestra.run.smithi043.stdout:checking malloc.h usability... yes 2024-04-26T20:29:33.067 INFO:teuthology.orchestra.run.smithi043.stdout:checking malloc.h presence... yes 2024-04-26T20:29:33.067 INFO:teuthology.orchestra.run.smithi043.stdout:checking for malloc.h... yes 2024-04-26T20:29:33.156 INFO:teuthology.orchestra.run.smithi043.stdout:checking uuid.h usability... no 2024-04-26T20:29:33.199 INFO:teuthology.orchestra.run.smithi043.stdout:checking uuid.h presence... no 2024-04-26T20:29:33.199 INFO:teuthology.orchestra.run.smithi043.stdout:checking for uuid.h... no 2024-04-26T20:29:33.292 INFO:teuthology.orchestra.run.smithi043.stdout:checking uuid/uuid.h usability... yes 2024-04-26T20:29:33.327 INFO:teuthology.orchestra.run.smithi043.stdout:checking uuid/uuid.h presence... yes 2024-04-26T20:29:33.327 INFO:teuthology.orchestra.run.smithi043.stdout:checking for uuid/uuid.h... yes 2024-04-26T20:29:33.411 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/uuid.h usability... no 2024-04-26T20:29:33.455 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/uuid.h presence... no 2024-04-26T20:29:33.455 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/uuid.h... no 2024-04-26T20:29:33.531 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/file.h usability... yes 2024-04-26T20:29:33.561 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/file.h presence... yes 2024-04-26T20:29:33.561 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/file.h... yes 2024-04-26T20:29:33.639 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fcntl.h usability... yes 2024-04-26T20:29:33.668 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fcntl.h presence... yes 2024-04-26T20:29:33.668 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/fcntl.h... yes 2024-04-26T20:29:33.751 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/syssgi.h usability... no 2024-04-26T20:29:33.791 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/syssgi.h presence... no 2024-04-26T20:29:33.791 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/syssgi.h... no 2024-04-26T20:29:33.887 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/param.h usability... yes 2024-04-26T20:29:33.928 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/param.h presence... yes 2024-04-26T20:29:33.928 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/param.h... yes 2024-04-26T20:29:33.946 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/stat.h... (cached) yes 2024-04-26T20:29:34.023 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/statvfs.h usability... yes 2024-04-26T20:29:34.054 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/statvfs.h presence... yes 2024-04-26T20:29:34.054 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/statvfs.h... yes 2024-04-26T20:29:34.138 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/time.h usability... yes 2024-04-26T20:29:34.166 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/time.h presence... yes 2024-04-26T20:29:34.166 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/time.h... yes 2024-04-26T20:29:34.257 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/ioctl.h usability... yes 2024-04-26T20:29:34.288 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/ioctl.h presence... yes 2024-04-26T20:29:34.288 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/ioctl.h... yes 2024-04-26T20:29:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/wait.h usability... yes 2024-04-26T20:29:34.415 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/wait.h presence... yes 2024-04-26T20:29:34.416 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/wait.h... yes 2024-04-26T20:29:34.437 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/types.h... (cached) yes 2024-04-26T20:29:34.455 INFO:teuthology.orchestra.run.smithi043.stdout:checking for strings.h... (cached) yes 2024-04-26T20:29:34.534 INFO:teuthology.orchestra.run.smithi043.stdout:checking err.h usability... yes 2024-04-26T20:29:34.562 INFO:teuthology.orchestra.run.smithi043.stdout:checking err.h presence... yes 2024-04-26T20:29:34.562 INFO:teuthology.orchestra.run.smithi043.stdout:checking for err.h... yes 2024-04-26T20:29:34.656 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/falloc.h usability... yes 2024-04-26T20:29:34.686 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/falloc.h presence... yes 2024-04-26T20:29:34.686 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/falloc.h... yes 2024-04-26T20:29:34.769 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-26T20:29:34.814 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-26T20:29:34.814 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-26T20:29:34.890 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-26T20:29:34.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-26T20:29:34.936 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-26T20:29:35.011 INFO:teuthology.orchestra.run.smithi043.stdout:checking xfs/platform_defs.h usability... no 2024-04-26T20:29:35.054 INFO:teuthology.orchestra.run.smithi043.stdout:checking xfs/platform_defs.h presence... no 2024-04-26T20:29:35.055 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/platform_defs.h... no 2024-04-26T20:29:35.133 INFO:teuthology.orchestra.run.smithi043.stdout:checking btrfs/ioctl.h usability... no 2024-04-26T20:29:35.173 INFO:teuthology.orchestra.run.smithi043.stdout:checking btrfs/ioctl.h presence... no 2024-04-26T20:29:35.173 INFO:teuthology.orchestra.run.smithi043.stdout:checking for btrfs/ioctl.h... no 2024-04-26T20:29:35.244 INFO:teuthology.orchestra.run.smithi043.stdout:checking cifs/ioctl.h usability... no 2024-04-26T20:29:35.288 INFO:teuthology.orchestra.run.smithi043.stdout:checking cifs/ioctl.h presence... no 2024-04-26T20:29:35.288 INFO:teuthology.orchestra.run.smithi043.stdout:checking for cifs/ioctl.h... no 2024-04-26T20:29:35.369 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/mman.h usability... yes 2024-04-26T20:29:35.398 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/mman.h presence... yes 2024-04-26T20:29:35.398 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/mman.h... yes 2024-04-26T20:29:35.477 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/ext4.h usability... no 2024-04-26T20:29:35.521 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/ext4.h presence... no 2024-04-26T20:29:35.521 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/ext4.h... no 2024-04-26T20:29:35.571 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/xfs_log_format.h... no 2024-04-26T20:29:35.681 INFO:teuthology.orchestra.run.smithi043.stdout:checking for uuid_compare... no 2024-04-26T20:29:35.761 INFO:teuthology.orchestra.run.smithi043.stdout:checking for uuid_compare in -luuid... yes 2024-04-26T20:29:35.873 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/xfs.h... yes 2024-04-26T20:29:35.925 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/libxfs.h... no 2024-04-26T20:29:35.990 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-26T20:29:36.068 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/xqm.h... yes 2024-04-26T20:29:36.187 INFO:teuthology.orchestra.run.smithi043.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-26T20:29:36.255 INFO:teuthology.orchestra.run.smithi043.stdout:checking xfs/handle.h usability... yes 2024-04-26T20:29:36.283 INFO:teuthology.orchestra.run.smithi043.stdout:checking xfs/handle.h presence... yes 2024-04-26T20:29:36.283 INFO:teuthology.orchestra.run.smithi043.stdout:checking for xfs/handle.h... yes 2024-04-26T20:29:36.378 INFO:teuthology.orchestra.run.smithi043.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-26T20:29:36.451 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/acl.h usability... yes 2024-04-26T20:29:36.485 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/acl.h presence... yes 2024-04-26T20:29:36.485 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/acl.h... yes 2024-04-26T20:29:36.566 INFO:teuthology.orchestra.run.smithi043.stdout:checking acl/libacl.h usability... yes 2024-04-26T20:29:36.597 INFO:teuthology.orchestra.run.smithi043.stdout:checking acl/libacl.h presence... yes 2024-04-26T20:29:36.597 INFO:teuthology.orchestra.run.smithi043.stdout:checking for acl/libacl.h... yes 2024-04-26T20:29:36.684 INFO:teuthology.orchestra.run.smithi043.stdout:checking for acl_init in -lacl... yes 2024-04-26T20:29:36.747 INFO:teuthology.orchestra.run.smithi043.stdout:checking gdbm-ndbm.h usability... no 2024-04-26T20:29:36.789 INFO:teuthology.orchestra.run.smithi043.stdout:checking gdbm-ndbm.h presence... no 2024-04-26T20:29:36.789 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gdbm-ndbm.h... no 2024-04-26T20:29:36.853 INFO:teuthology.orchestra.run.smithi043.stdout:checking gdbm/ndbm.h usability... yes 2024-04-26T20:29:36.886 INFO:teuthology.orchestra.run.smithi043.stdout:checking gdbm/ndbm.h presence... yes 2024-04-26T20:29:36.886 INFO:teuthology.orchestra.run.smithi043.stdout:checking for gdbm/ndbm.h... yes 2024-04-26T20:29:36.993 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_open in -lgdbm... no 2024-04-26T20:29:37.093 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-26T20:29:37.195 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_store in -lgdbm... no 2024-04-26T20:29:37.297 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_close in -lgdbm... no 2024-04-26T20:29:37.397 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-26T20:29:37.483 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-26T20:29:37.575 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-26T20:29:37.665 INFO:teuthology.orchestra.run.smithi043.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-26T20:29:37.733 INFO:teuthology.orchestra.run.smithi043.stdout:checking libaio.h usability... yes 2024-04-26T20:29:37.767 INFO:teuthology.orchestra.run.smithi043.stdout:checking libaio.h presence... yes 2024-04-26T20:29:37.767 INFO:teuthology.orchestra.run.smithi043.stdout:checking for libaio.h... yes 2024-04-26T20:29:37.836 INFO:teuthology.orchestra.run.smithi043.stdout:checking liburing.h usability... no 2024-04-26T20:29:37.881 INFO:teuthology.orchestra.run.smithi043.stdout:checking liburing.h presence... no 2024-04-26T20:29:37.881 INFO:teuthology.orchestra.run.smithi043.stdout:checking for liburing.h... no 2024-04-26T20:29:37.955 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/capability.h usability... yes 2024-04-26T20:29:37.989 INFO:teuthology.orchestra.run.smithi043.stdout:checking sys/capability.h presence... yes 2024-04-26T20:29:37.989 INFO:teuthology.orchestra.run.smithi043.stdout:checking for sys/capability.h... yes 2024-04-26T20:29:38.060 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/fiemap.h usability... yes 2024-04-26T20:29:38.091 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/fiemap.h presence... yes 2024-04-26T20:29:38.091 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/fiemap.h... yes 2024-04-26T20:29:38.186 INFO:teuthology.orchestra.run.smithi043.stdout:checking for fallocate... yes 2024-04-26T20:29:38.279 INFO:teuthology.orchestra.run.smithi043.stdout:checking for open_by_handle_at... yes 2024-04-26T20:29:38.349 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/fs.h usability... yes 2024-04-26T20:29:38.381 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/fs.h presence... yes 2024-04-26T20:29:38.381 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/fs.h... yes 2024-04-26T20:29:38.450 INFO:teuthology.orchestra.run.smithi043.stdout:checking btrfsutil.h usability... no 2024-04-26T20:29:38.496 INFO:teuthology.orchestra.run.smithi043.stdout:checking btrfsutil.h presence... no 2024-04-26T20:29:38.496 INFO:teuthology.orchestra.run.smithi043.stdout:checking for btrfsutil.h... no 2024-04-26T20:29:38.594 INFO:teuthology.orchestra.run.smithi043.stdout:checking for copy_file_range... yes 2024-04-26T20:29:38.691 INFO:teuthology.orchestra.run.smithi043.stdout:checking for SEEK_DATA... yes 2024-04-26T20:29:38.813 INFO:teuthology.orchestra.run.smithi043.stdout:checking for BMV_OF_SHARED... yes 2024-04-26T20:29:38.915 INFO:teuthology.orchestra.run.smithi043.stdout:checking for nftw... yes 2024-04-26T20:29:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:checking for RLIMIT_NOFILE... yes 2024-04-26T20:29:39.101 INFO:teuthology.orchestra.run.smithi043.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-26T20:29:39.196 INFO:teuthology.orchestra.run.smithi043.stdout:checking for FICLONE... yes 2024-04-26T20:29:39.290 INFO:teuthology.orchestra.run.smithi043.stdout:checking for renameat2... yes 2024-04-26T20:29:39.381 INFO:teuthology.orchestra.run.smithi043.stdout:checking for reallocarray... yes 2024-04-26T20:29:39.483 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct mount_attr... yes 2024-04-26T20:29:39.600 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-26T20:29:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-26T20:29:39.824 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-26T20:29:39.939 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-26T20:29:40.053 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-26T20:29:40.152 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-26T20:29:40.270 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-26T20:29:40.342 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/btrfs.h usability... yes 2024-04-26T20:29:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/btrfs.h presence... yes 2024-04-26T20:29:40.370 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/btrfs.h... yes 2024-04-26T20:29:40.455 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-26T20:29:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-26T20:29:40.486 INFO:teuthology.orchestra.run.smithi043.stdout:checking for linux/btrfs_tree.h... yes 2024-04-26T20:29:40.555 INFO:teuthology.orchestra.run.smithi043.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-26T20:29:40.625 INFO:teuthology.orchestra.run.smithi043.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-26T20:29:40.664 INFO:teuthology.orchestra.run.smithi043.stdout:configure: creating ./config.status 2024-04-26T20:29:41.898 INFO:teuthology.orchestra.run.smithi043.stdout:config.status: creating include/builddefs 2024-04-26T20:29:41.945 INFO:teuthology.orchestra.run.smithi043.stdout:config.status: creating include/config.h 2024-04-26T20:29:41.977 INFO:teuthology.orchestra.run.smithi043.stdout:config.status: executing libtool commands 2024-04-26T20:29:42.079 INFO:teuthology.orchestra.run.smithi043.stdout:Building include 2024-04-26T20:29:42.089 INFO:teuthology.orchestra.run.smithi043.stdout:Building lib 2024-04-26T20:29:42.098 INFO:teuthology.orchestra.run.smithi043.stdout: [LTDEP] 2024-04-26T20:29:42.443 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dataascii.lo 2024-04-26T20:29:42.738 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] databin.lo 2024-04-26T20:29:43.055 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] datapid.lo 2024-04-26T20:29:43.318 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] file_lock.lo 2024-04-26T20:29:43.651 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] forker.lo 2024-04-26T20:29:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] pattern.lo 2024-04-26T20:29:44.228 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] open_flags.lo 2024-04-26T20:29:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] random_range.lo 2024-04-26T20:29:44.936 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] string_to_tokens.lo 2024-04-26T20:29:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] str_to_bytes.lo 2024-04-26T20:29:45.501 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] tlibio.lo 2024-04-26T20:29:45.923 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] write_log.lo 2024-04-26T20:29:46.346 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] random.lo 2024-04-26T20:29:46.615 INFO:teuthology.orchestra.run.smithi043.stdout: [LD] 2024-04-26T20:29:46.832 INFO:teuthology.orchestra.run.smithi043.stdout:Building ltp 2024-04-26T20:29:46.842 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:47.121 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] doio 2024-04-26T20:29:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fsstress 2024-04-26T20:29:49.783 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fsx 2024-04-26T20:29:50.781 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] growfiles 2024-04-26T20:29:51.516 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] iogen 2024-04-26T20:29:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-stress 2024-04-26T20:29:52.759 INFO:teuthology.orchestra.run.smithi043.stdout:Building src 2024-04-26T20:29:52.774 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:56.848 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:56.926 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:56.948 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:57.464 INFO:teuthology.orchestra.run.smithi043.stdout: [DEP] 2024-04-26T20:29:57.521 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dirstress 2024-04-26T20:29:57.973 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fill 2024-04-26T20:29:58.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:29:58.057 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:29:58.283 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fill2 2024-04-26T20:29:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] getpagesize 2024-04-26T20:29:59.037 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] holes 2024-04-26T20:29:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] lstat64 2024-04-26T20:29:59.792 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] nametest 2024-04-26T20:30:00.272 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] permname 2024-04-26T20:30:00.662 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] randholes 2024-04-26T20:30:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] runas 2024-04-26T20:30:01.491 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] truncfile 2024-04-26T20:30:01.878 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] usemem 2024-04-26T20:30:02.219 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] mmapcat 2024-04-26T20:30:02.546 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] append_reader 2024-04-26T20:30:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] append_writer 2024-04-26T20:30:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dirperf 2024-04-26T20:30:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] metaperf 2024-04-26T20:30:04.178 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] devzero 2024-04-26T20:30:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] feature 2024-04-26T20:30:04.984 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] alloc 2024-04-26T20:30:05.381 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fault 2024-04-26T20:30:05.717 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fstest 2024-04-26T20:30:06.164 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_access_root 2024-04-26T20:30:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] godown 2024-04-26T20:30:06.848 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] resvtest 2024-04-26T20:30:07.235 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] writemod 2024-04-26T20:30:07.571 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] writev_on_pagefault 2024-04-26T20:30:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] makeextents 2024-04-26T20:30:08.290 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] itrash 2024-04-26T20:30:08.639 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] rename 2024-04-26T20:30:08.934 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] multi_open_unlink 2024-04-26T20:30:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] unwritten_sync 2024-04-26T20:30:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] genhashnames 2024-04-26T20:30:10.083 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_holes 2024-04-26T20:30:10.410 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_writev 2024-04-26T20:30:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_truncate_cmtime 2024-04-26T20:30:11.107 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dirhash_collide 2024-04-26T20:30:11.467 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_rename_overwrite 2024-04-26T20:30:11.781 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] holetest 2024-04-26T20:30:12.201 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_truncate_self 2024-04-26T20:30:12.551 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] af_unix 2024-04-26T20:30:12.893 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_stale_pmd 2024-04-26T20:30:13.231 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_cow_race 2024-04-26T20:30:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_fallocate 2024-04-26T20:30:13.942 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fsync-err 2024-04-26T20:30:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_write_ro 2024-04-26T20:30:14.661 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-26T20:30:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-26T20:30:15.390 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_ofd_locks 2024-04-26T20:30:15.780 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_collision 2024-04-26T20:30:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] mmap-write-concurrent 2024-04-26T20:30:16.510 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_get_file_time 2024-04-26T20:30:16.858 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_create_short_dirs 2024-04-26T20:30:17.196 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_create_long_dirs 2024-04-26T20:30:17.571 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_enospc 2024-04-26T20:30:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_writev_overlap 2024-04-26T20:30:18.319 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] checkpoint_journal 2024-04-26T20:30:18.659 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] mmap-rw-fault 2024-04-26T20:30:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] allocstale 2024-04-26T20:30:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_cow_memory_failure 2024-04-26T20:30:19.774 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fake-dump-rootino 2024-04-26T20:30:20.140 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dio-buf-fault 2024-04-26T20:30:20.483 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] rewinddir-test 2024-04-26T20:30:20.853 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] readdir-while-renames 2024-04-26T20:30:21.200 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] xfsctl 2024-04-26T20:30:21.566 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] bstat 2024-04-26T20:30:21.971 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mtab 2024-04-26T20:30:22.380 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] getdevicesize 2024-04-26T20:30:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] preallo_rw_pattern_reader 2024-04-26T20:30:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] preallo_rw_pattern_writer 2024-04-26T20:30:23.456 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] ftrunc 2024-04-26T20:30:23.761 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] trunc 2024-04-26T20:30:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fs_perms 2024-04-26T20:30:24.433 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] testx 2024-04-26T20:30:24.719 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] looptest 2024-04-26T20:30:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] locktest 2024-04-26T20:30:25.612 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] unwritten_mmap 2024-04-26T20:30:25.981 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] bulkstat_unlink_test 2024-04-26T20:30:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] deduperace 2024-04-26T20:30:26.771 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] bulkstat_unlink_test_modified 2024-04-26T20:30:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_dir_offset 2024-04-26T20:30:27.523 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_futimens 2024-04-26T20:30:27.859 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_immutable 2024-04-26T20:30:28.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:30:28.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:30:28.712 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] stale_handle 2024-04-26T20:30:29.088 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] pwrite_mmap_blocked 2024-04-26T20:30:29.476 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_dir_offset2 2024-04-26T20:30:29.873 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] seek_sanity_test 2024-04-26T20:30:30.527 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] seek_copy_test 2024-04-26T20:30:30.917 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_readdir_1 2024-04-26T20:30:31.226 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_readdir_2 2024-04-26T20:30:31.542 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fsync-tester 2024-04-26T20:30:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] nsexec 2024-04-26T20:30:32.376 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] cloner 2024-04-26T20:30:32.760 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] renameat2 2024-04-26T20:30:33.145 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_getcwd 2024-04-26T20:30:33.500 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] e4compact 2024-04-26T20:30:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] test-nextquota 2024-04-26T20:30:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] punch-alternating 2024-04-26T20:30:34.638 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-26T20:30:34.978 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] listxattr 2024-04-26T20:30:35.333 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dio-interleaved 2024-04-26T20:30:35.699 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_dir_type 2024-04-26T20:30:36.043 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dio-invalidate-cache 2024-04-26T20:30:36.453 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] stat_test 2024-04-26T20:30:36.958 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_encrypted_d_revalidate 2024-04-26T20:30:37.313 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] attr_replace_test 2024-04-26T20:30:37.673 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] swapon 2024-04-26T20:30:38.023 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] mkswap 2024-04-26T20:30:38.393 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_attr_corruption 2024-04-26T20:30:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_open_tmpfiles 2024-04-26T20:30:39.121 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fscrypt-crypt-util 2024-04-26T20:30:40.031 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] bulkstat_null_ocount 2024-04-26T20:30:40.380 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] splice-test 2024-04-26T20:30:40.767 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] chprojid_fail 2024-04-26T20:30:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] detached_mounts_propagation 2024-04-26T20:30:41.481 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] ext4_resize 2024-04-26T20:30:41.833 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_readdir_3 2024-04-26T20:30:42.234 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] splice2pipe 2024-04-26T20:30:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] uuid_ioctl 2024-04-26T20:30:42.975 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-26T20:30:43.344 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fiemap-fault 2024-04-26T20:30:43.675 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fiemap-tester 2024-04-26T20:30:44.167 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_stripealign 2024-04-26T20:30:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] open_by_handle 2024-04-26T20:30:45.000 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] dbtest 2024-04-26T20:30:45.428 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_mmap_dio 2024-04-26T20:30:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] xfsfind 2024-04-26T20:30:46.171 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] t_reflink_read_race 2024-04-26T20:30:46.550 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] fssum 2024-04-26T20:30:47.219 INFO:teuthology.orchestra.run.smithi043.stdout:Building log-writes 2024-04-26T20:30:47.233 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] replay-log 2024-04-26T20:30:47.771 INFO:teuthology.orchestra.run.smithi043.stdout:Building perf 2024-04-26T20:30:47.782 INFO:teuthology.orchestra.run.smithi043.stdout:Building aio-dio-regress 2024-04-26T20:30:47.798 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aiocp 2024-04-26T20:30:48.208 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-26T20:30:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-append-write-read-race 2024-04-26T20:30:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-cow-race 2024-04-26T20:30:49.261 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-cycle-write 2024-04-26T20:30:49.625 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-eof-race 2024-04-26T20:30:50.008 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-extend-stat 2024-04-26T20:30:50.354 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-fcntl-race 2024-04-26T20:30:50.703 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-hole-filling-race 2024-04-26T20:30:51.018 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-invalidate-failure 2024-04-26T20:30:51.353 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-invalidate-readahead 2024-04-26T20:30:51.693 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aiodio_sparse2 2024-04-26T20:30:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-subblock-eof-read 2024-04-26T20:30:52.433 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-dio-write-verify 2024-04-26T20:30:52.822 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-26T20:30:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-26T20:30:53.445 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] aio-last-ref-held-by-io 2024-04-26T20:30:53.842 INFO:teuthology.orchestra.run.smithi043.stdout:Building vfs 2024-04-26T20:30:53.858 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] vfstest 2024-04-26T20:30:57.982 INFO:teuthology.orchestra.run.smithi043.stdout: [CC] mount-idmapped 2024-04-26T20:30:58.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:30:58.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:30:58.619 INFO:teuthology.orchestra.run.smithi043.stdout:Building m4 2024-04-26T20:30:58.630 INFO:teuthology.orchestra.run.smithi043.stdout:Building common 2024-04-26T20:30:58.640 INFO:teuthology.orchestra.run.smithi043.stdout:Building tests 2024-04-26T20:30:58.655 INFO:teuthology.orchestra.run.smithi043.stdout:Building btrfs 2024-04-26T20:30:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/btrfs/group.list 2024-04-26T20:30:58.909 INFO:teuthology.orchestra.run.smithi043.stdout:Building ceph 2024-04-26T20:30:58.924 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ceph/group.list 2024-04-26T20:30:58.981 INFO:teuthology.orchestra.run.smithi043.stdout:Building cifs 2024-04-26T20:30:58.996 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/cifs/group.list 2024-04-26T20:30:59.044 INFO:teuthology.orchestra.run.smithi043.stdout:Building ext4 2024-04-26T20:30:59.060 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ext4/group.list 2024-04-26T20:30:59.211 INFO:teuthology.orchestra.run.smithi043.stdout:Building f2fs 2024-04-26T20:30:59.227 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/f2fs/group.list 2024-04-26T20:30:59.294 INFO:teuthology.orchestra.run.smithi043.stdout:Building generic 2024-04-26T20:30:59.314 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/generic/group.list 2024-04-26T20:30:59.680 INFO:teuthology.orchestra.run.smithi043.stdout:Building nfs 2024-04-26T20:30:59.697 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/nfs/group.list 2024-04-26T20:30:59.754 INFO:teuthology.orchestra.run.smithi043.stdout:Building ocfs2 2024-04-26T20:30:59.770 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ocfs2/group.list 2024-04-26T20:30:59.824 INFO:teuthology.orchestra.run.smithi043.stdout:Building overlay 2024-04-26T20:30:59.839 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/overlay/group.list 2024-04-26T20:31:00.013 INFO:teuthology.orchestra.run.smithi043.stdout:Building perf 2024-04-26T20:31:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/perf/group.list 2024-04-26T20:31:00.075 INFO:teuthology.orchestra.run.smithi043.stdout:Building selftest 2024-04-26T20:31:00.091 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/selftest/group.list 2024-04-26T20:31:00.139 INFO:teuthology.orchestra.run.smithi043.stdout:Building shared 2024-04-26T20:31:00.154 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/shared/group.list 2024-04-26T20:31:00.222 INFO:teuthology.orchestra.run.smithi043.stdout:Building tmpfs 2024-04-26T20:31:00.237 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/tmpfs/group.list 2024-04-26T20:31:00.290 INFO:teuthology.orchestra.run.smithi043.stdout:Building udf 2024-04-26T20:31:00.306 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/udf/group.list 2024-04-26T20:31:00.350 INFO:teuthology.orchestra.run.smithi043.stdout:Building xfs 2024-04-26T20:31:00.369 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/xfs/group.list 2024-04-26T20:31:00.763 DEBUG:teuthology.orchestra.run.smithi043:> (cd /tmp/tmp.dmSIwu2Lebxfstests-dev && exec sudo make install) 2024-04-26T20:31:00.883 INFO:teuthology.orchestra.run.smithi043.stdout:Building include 2024-04-26T20:31:00.894 INFO:teuthology.orchestra.run.smithi043.stdout:Building lib 2024-04-26T20:31:00.907 INFO:teuthology.orchestra.run.smithi043.stdout:Building ltp 2024-04-26T20:31:00.919 INFO:teuthology.orchestra.run.smithi043.stdout:Building src 2024-04-26T20:31:01.010 INFO:teuthology.orchestra.run.smithi043.stdout:Building log-writes 2024-04-26T20:31:01.022 INFO:teuthology.orchestra.run.smithi043.stdout:Building perf 2024-04-26T20:31:01.033 INFO:teuthology.orchestra.run.smithi043.stdout:Building aio-dio-regress 2024-04-26T20:31:01.045 INFO:teuthology.orchestra.run.smithi043.stdout:Building vfs 2024-04-26T20:31:01.057 INFO:teuthology.orchestra.run.smithi043.stdout:Building m4 2024-04-26T20:31:01.069 INFO:teuthology.orchestra.run.smithi043.stdout:Building common 2024-04-26T20:31:01.080 INFO:teuthology.orchestra.run.smithi043.stdout:Building tests 2024-04-26T20:31:01.095 INFO:teuthology.orchestra.run.smithi043.stdout:Building btrfs 2024-04-26T20:31:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/btrfs/group.list 2024-04-26T20:31:01.339 INFO:teuthology.orchestra.run.smithi043.stdout:Building ceph 2024-04-26T20:31:01.353 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ceph/group.list 2024-04-26T20:31:01.406 INFO:teuthology.orchestra.run.smithi043.stdout:Building cifs 2024-04-26T20:31:01.421 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/cifs/group.list 2024-04-26T20:31:01.470 INFO:teuthology.orchestra.run.smithi043.stdout:Building ext4 2024-04-26T20:31:01.486 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ext4/group.list 2024-04-26T20:31:01.635 INFO:teuthology.orchestra.run.smithi043.stdout:Building f2fs 2024-04-26T20:31:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/f2fs/group.list 2024-04-26T20:31:01.719 INFO:teuthology.orchestra.run.smithi043.stdout:Building generic 2024-04-26T20:31:01.738 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/generic/group.list 2024-04-26T20:31:02.145 INFO:teuthology.orchestra.run.smithi043.stdout:Building nfs 2024-04-26T20:31:02.163 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/nfs/group.list 2024-04-26T20:31:02.219 INFO:teuthology.orchestra.run.smithi043.stdout:Building ocfs2 2024-04-26T20:31:02.234 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ocfs2/group.list 2024-04-26T20:31:02.287 INFO:teuthology.orchestra.run.smithi043.stdout:Building overlay 2024-04-26T20:31:02.302 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/overlay/group.list 2024-04-26T20:31:02.467 INFO:teuthology.orchestra.run.smithi043.stdout:Building perf 2024-04-26T20:31:02.484 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/perf/group.list 2024-04-26T20:31:02.528 INFO:teuthology.orchestra.run.smithi043.stdout:Building selftest 2024-04-26T20:31:02.543 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/selftest/group.list 2024-04-26T20:31:02.592 INFO:teuthology.orchestra.run.smithi043.stdout:Building shared 2024-04-26T20:31:02.605 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/shared/group.list 2024-04-26T20:31:02.671 INFO:teuthology.orchestra.run.smithi043.stdout:Building tmpfs 2024-04-26T20:31:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/tmpfs/group.list 2024-04-26T20:31:02.740 INFO:teuthology.orchestra.run.smithi043.stdout:Building udf 2024-04-26T20:31:02.756 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/udf/group.list 2024-04-26T20:31:02.800 INFO:teuthology.orchestra.run.smithi043.stdout:Building xfs 2024-04-26T20:31:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/xfs/group.list 2024-04-26T20:31:03.220 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-26T20:31:03.225 INFO:teuthology.orchestra.run.smithi043.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-26T20:31:03.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-26T20:31:03.230 INFO:teuthology.orchestra.run.smithi043.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-26T20:31:03.230 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-26T20:31:03.236 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-26T20:31:03.271 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-26T20:31:03.525 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-26T20:31:03.562 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-26T20:31:03.609 INFO:teuthology.orchestra.run.smithi043.stdout:Building log-writes 2024-04-26T20:31:03.620 INFO:teuthology.orchestra.run.smithi043.stdout:Building perf 2024-04-26T20:31:03.631 INFO:teuthology.orchestra.run.smithi043.stdout:Building aio-dio-regress 2024-04-26T20:31:03.643 INFO:teuthology.orchestra.run.smithi043.stdout:Building vfs 2024-04-26T20:31:03.652 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake -C log-writes install 2024-04-26T20:31:03.655 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-26T20:31:03.690 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-26T20:31:03.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake -C perf install 2024-04-26T20:31:03.729 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-26T20:31:03.764 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-26T20:31:03.852 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-26T20:31:03.857 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-26T20:31:03.892 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-26T20:31:04.096 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake -C vfs install 2024-04-26T20:31:04.100 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-26T20:31:04.135 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-26T20:31:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-26T20:31:04.218 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-26T20:31:09.008 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-26T20:31:09.372 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-26T20:31:09.449 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-26T20:31:09.454 INFO:teuthology.orchestra.run.smithi043.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-26T20:31:09.454 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-26T20:31:09.459 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-26T20:31:09.493 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-26T20:31:10.039 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-26T20:31:10.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/btrfs/ install 2024-04-26T20:31:10.051 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/btrfs/group.list 2024-04-26T20:31:10.285 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-26T20:31:10.320 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-26T20:31:14.056 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-26T20:31:14.092 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-26T20:31:17.955 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ceph/ install 2024-04-26T20:31:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ceph/group.list 2024-04-26T20:31:18.016 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-26T20:31:18.048 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-26T20:31:18.125 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-26T20:31:18.159 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-26T20:31:18.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/cifs/ install 2024-04-26T20:31:18.241 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/cifs/group.list 2024-04-26T20:31:18.286 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-26T20:31:18.316 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-26T20:31:18.352 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-26T20:31:18.386 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-26T20:31:18.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ext4/ install 2024-04-26T20:31:18.429 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ext4/group.list 2024-04-26T20:31:18.575 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-26T20:31:18.608 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-26T20:31:19.339 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-26T20:31:19.372 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-26T20:31:20.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/f2fs/ install 2024-04-26T20:31:20.168 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/f2fs/group.list 2024-04-26T20:31:20.233 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-26T20:31:20.267 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-26T20:31:20.311 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-26T20:31:20.345 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-26T20:31:20.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/generic/ install 2024-04-26T20:31:20.399 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/generic/group.list 2024-04-26T20:31:20.773 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-26T20:31:20.808 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-26T20:31:28.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:31:28.462 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:31:29.470 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-26T20:31:29.498 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-26T20:31:38.242 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/nfs/ install 2024-04-26T20:31:38.248 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/nfs/group.list 2024-04-26T20:31:38.301 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-26T20:31:38.335 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-26T20:31:38.370 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-26T20:31:38.406 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-26T20:31:38.441 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ocfs2/ install 2024-04-26T20:31:38.448 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/ocfs2/group.list 2024-04-26T20:31:38.496 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-26T20:31:38.531 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-26T20:31:38.567 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-26T20:31:38.601 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-26T20:31:38.637 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/overlay/ install 2024-04-26T20:31:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/overlay/group.list 2024-04-26T20:31:38.800 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-26T20:31:38.833 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-26T20:31:40.054 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-26T20:31:40.089 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-26T20:31:41.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/perf/ install 2024-04-26T20:31:41.326 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/perf/group.list 2024-04-26T20:31:41.369 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-26T20:31:41.403 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-26T20:31:41.436 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-26T20:31:41.469 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-26T20:31:41.505 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/selftest/ install 2024-04-26T20:31:41.512 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/selftest/group.list 2024-04-26T20:31:41.558 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-26T20:31:41.592 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-26T20:31:41.681 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-26T20:31:41.717 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-26T20:31:41.806 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/shared/ install 2024-04-26T20:31:41.814 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/shared/group.list 2024-04-26T20:31:41.878 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-26T20:31:41.910 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-26T20:31:41.966 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-26T20:31:41.999 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-26T20:31:42.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/tmpfs/ install 2024-04-26T20:31:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/tmpfs/group.list 2024-04-26T20:31:42.110 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-26T20:31:42.144 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-26T20:31:42.179 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-26T20:31:42.214 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-26T20:31:42.248 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/udf/ install 2024-04-26T20:31:42.256 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/udf/group.list 2024-04-26T20:31:42.298 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-26T20:31:42.331 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-26T20:31:42.364 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-26T20:31:42.398 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-26T20:31:42.434 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/xfs/ install 2024-04-26T20:31:42.447 INFO:teuthology.orchestra.run.smithi043.stdout: [GROUP] /tmp/tmp.dmSIwu2Lebxfstests-dev/tests/xfs/group.list 2024-04-26T20:31:42.839 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-26T20:31:42.875 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-26T20:31:51.213 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-26T20:31:51.248 INFO:teuthology.orchestra.run.smithi043.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-26T20:31:58.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:31:58.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:31:59.655 INFO:teuthology.orchestra.run.smithi043.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-26T20:31:59.691 INFO:teuthology.orchestra.run.smithi043.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-26T20:31:59.728 INFO:teuthology.orchestra.run.smithi043.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-26T20:31:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-26T20:31:59.804 INFO:tasks.cephfs.test_acls:client is fuse mounted 2024-04-26T20:31:59.804 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-26T20:31:59.805 DEBUG:teuthology.orchestra.run.smithi043:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.dmSIwu2Lebxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-26T20:32:00.334 INFO:teuthology.orchestra.run.smithi043.stdout:FSTYP -- ceph-fuse 2024-04-26T20:32:00.346 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM -- Linux/x86_64 smithi043 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-26T20:32:00.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:01.472 INFO:teuthology.orchestra.run.smithi043.stdout:generic/099 1s 2024-04-26T20:32:01.491 INFO:teuthology.orchestra.run.smithi043.stdout:Ran: generic/099 2024-04-26T20:32:01.491 INFO:teuthology.orchestra.run.smithi043.stdout:Passed all 1 tests 2024-04-26T20:32:01.500 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:01.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48756]: fuse finished with error 0 and tester_r 0 2024-04-26T20:32:01.530 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-04-26T20:32:01.530 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph-fuse PLATFORM -- Linux/x86_64 smithi043 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 generic/099 1s Ran: generic/099 Passed all 1 tests 2024-04-26T20:32:01.530 INFO:tasks.cephfs.test_acls:Command stderr - 2024-04-26T20:32:01.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo userdel --force --remove fsgqa 2024-04-26T20:32:02.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo userdel --force --remove 123456-fsgqa 2024-04-26T20:32:02.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo groupdel fsgqa 2024-04-26T20:32:02.968 INFO:teuthology.orchestra.run.smithi043.stderr:groupdel: group 'fsgqa' does not exist 2024-04-26T20:32:02.972 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-26T20:32:02.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:02.973 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.dmSIwu2Lebxfstests-dev/results) 2024-04-26T20:32:03.026 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-26T20:32:03.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:32:03.027 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.dmSIwu2Lebxfstests-dev 2024-04-26T20:32:03.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf /tmp/tmp.dmSIwu2Lebxfstests-dev 2024-04-26T20:32:03.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:32:03.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:32:03.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:32:03.572 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:32:03.572 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:32:03.573 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:32:03.573 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:03.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:32:03.630 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:32:03.630 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:32:03.630 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:03.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:32:03.688 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:32:03.688 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:32:03.688 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:03.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:32:03.749 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:32:03.750 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:32:03.750 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:03.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:32:03.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:32:04.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.034+0000 7fdf9da00640 1 Processor -- start 2024-04-26T20:32:04.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.034+0000 7fdf9da00640 1 -- start start 2024-04-26T20:32:04.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.036+0000 7fdf9da00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf98072620 0x7fdf980cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.036+0000 7fdf9da00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 0x7fdf9806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.036+0000 7fdf9da00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf98072a80 con 0x7fdf980cd180 2024-04-26T20:32:04.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.036+0000 7fdf9da00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdf980ce6e0 con 0x7fdf98072620 2024-04-26T20:32:04.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.036+0000 7fdf9da00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdf980cd560 con 0x7fdf980ce2b0 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf98072620 0x7fdf980cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf98072620 0x7fdf980cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43780/0 (socket says 172.21.15.43:43780) 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 -- 172.21.15.43:0/1093030875 learned_addr learned my addr 172.21.15.43:0/1093030875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf9d000640 1 --2- 172.21.15.43:0/1093030875 >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 0x7fdf9806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 -- 172.21.15.43:0/1093030875 >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 msgr2=0x7fdf9806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 --2- 172.21.15.43:0/1093030875 >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 0x7fdf9806dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 -- 172.21.15.43:0/1093030875 >> v1:172.21.15.164:6789/0 conn(0x7fdf980cd180 legacy=0x7fdf9815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf8fe00640 1 -- 172.21.15.43:0/1093030875 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf98072d00 con 0x7fdf98072620 2024-04-26T20:32:04.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.037+0000 7fdf9d000640 1 --2- 172.21.15.43:0/1093030875 >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 0x7fdf9806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:04.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.038+0000 7fdf8fe00640 1 --2- 172.21.15.43:0/1093030875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98072620 0x7fdf980cdd70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdf980b39f0 tx=0x7fdf8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3752ae974d3eba57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.039+0000 7fdf97400640 1 -- 172.21.15.43:0/1093030875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80037070 con 0x7fdf98072620 2024-04-26T20:32:04.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.039+0000 7fdf97400640 1 -- 172.21.15.43:0/1093030875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf80032440 con 0x7fdf98072620 2024-04-26T20:32:04.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.039+0000 7fdf97400640 1 -- 172.21.15.43:0/1093030875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80032810 con 0x7fdf98072620 2024-04-26T20:32:04.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.039+0000 7fdf9da00640 1 -- 172.21.15.43:0/1093030875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98072620 msgr2=0x7fdf980cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.039+0000 7fdf9da00640 1 --2- 172.21.15.43:0/1093030875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98072620 0x7fdf980cdd70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdf980b39f0 tx=0x7fdf8002a230 comp rx=0 tx=0).stop 2024-04-26T20:32:04.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 -- 172.21.15.43:0/1093030875 shutdown_connections 2024-04-26T20:32:04.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 --2- 172.21.15.43:0/1093030875 >> v2:172.21.15.164:3300/0 conn(0x7fdf980ce2b0 0x7fdf9806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 --2- 172.21.15.43:0/1093030875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf98072620 0x7fdf980cdd70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 -- 172.21.15.43:0/1093030875 >> 172.21.15.43:0/1093030875 conn(0x7fdf98068090 msgr2=0x7fdf98068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:04.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 -- 172.21.15.43:0/1093030875 shutdown_connections 2024-04-26T20:32:04.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.040+0000 7fdf9da00640 1 -- 172.21.15.43:0/1093030875 wait complete. 2024-04-26T20:32:04.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.042+0000 7fdf9da00640 1 Processor -- start 2024-04-26T20:32:04.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.042+0000 7fdf9da00640 1 -- start start 2024-04-26T20:32:04.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98072620 0x7fdf9813acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf980cd180 0x7fdf98140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf980735a0 con 0x7fdf980ce2b0 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf980732c0 con 0x7fdf980cd180 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9da00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf98073da0 con 0x7fdf98072620 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43788/0 (socket says 172.21.15.43:43788) 2024-04-26T20:32:04.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.043+0000 7fdf9d000640 1 -- 172.21.15.43:0/475055031 learned_addr learned my addr 172.21.15.43:0/475055031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf97e00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98072620 0x7fdf9813acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 -- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98072620 msgr2=0x7fdf9813acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98072620 0x7fdf9813acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 -- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf980cd180 msgr2=0x7fdf98140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf980cd180 0x7fdf98140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 -- 172.21.15.43:0/475055031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9813b210 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9d000640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000cd50 tx=0x7fdf9000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000f680 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9000f840 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000fc30 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf981f3200 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.044+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981f3780 con 0x7fdf980ce2b0 2024-04-26T20:32:04.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.046+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf980aff20 con 0x7fdf980ce2b0 2024-04-26T20:32:04.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.047+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf9000e3e0 con 0x7fdf980ce2b0 2024-04-26T20:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.047+0000 7fdf94c00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.048+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fdf9008ca70 con 0x7fdf980ce2b0 2024-04-26T20:32:04.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.048+0000 7fdf97e00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.048+0000 7fdf97e00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 0x7fdf7406ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf88003d90 tx=0x7fdf88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.052+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf980aff20 con 0x7fdf980ce2b0 2024-04-26T20:32:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.297+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf980afce0 con 0x7fdf980ce2b0 2024-04-26T20:32:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.298+0000 7fdf94c00640 1 -- 172.21.15.43:0/475055031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fdf9813b210 con 0x7fdf980ce2b0 2024-04-26T20:32:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:04.300 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:32:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:32:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 msgr2=0x7fdf7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 0x7fdf7406ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf88003d90 tx=0x7fdf88009040 comp rx=0 tx=0).stop 2024-04-26T20:32:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 msgr2=0x7fdf98143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000cd50 tx=0x7fdf9000a650 comp rx=0 tx=0).stop 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 shutdown_connections 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf980ce2b0 0x7fdf98143a40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf980cd180 0x7fdf98140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 --2- 172.21.15.43:0/475055031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf98072620 0x7fdf9813acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 >> 172.21.15.43:0/475055031 conn(0x7fdf98068090 msgr2=0x7fdf980b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 shutdown_connections 2024-04-26T20:32:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.301+0000 7fdf9da00640 1 -- 172.21.15.43:0/475055031 wait complete. 2024-04-26T20:32:04.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:32:04.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.528+0000 7f9899400640 1 Processor -- start 2024-04-26T20:32:04.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.528+0000 7f9899400640 1 -- start start 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.529+0000 7f9899400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98940cd180 0x7f9894072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.529+0000 7f9899400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98940cd560 con 0x7f989415dc50 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.529+0000 7f9899400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9894072d00 con 0x7f98940ce480 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.529+0000 7f9899400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9894072fe0 con 0x7f98940cd180 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98940cd180 0x7f9894072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98940cd180 0x7f9894072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43804/0 (socket says 172.21.15.43:43804) 2024-04-26T20:32:04.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 -- 172.21.15.43:0/1119243051 learned_addr learned my addr 172.21.15.43:0/1119243051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 -- 172.21.15.43:0/1119243051 >> v1:172.21.15.164:6789/0 conn(0x7f98940ce480 legacy=0x7f989406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:04.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 -- 172.21.15.43:0/1119243051 >> v1:172.21.15.43:6789/0 conn(0x7f989415dc50 legacy=0x7f98940cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.530+0000 7f9893e00640 1 -- 172.21.15.43:0/1119243051 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98940732c0 con 0x7f98940cd180 2024-04-26T20:32:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.531+0000 7f9893e00640 1 --2- 172.21.15.43:0/1119243051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98940cd180 0x7f9894072620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f988402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378c0746b6342bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9892a00640 1 -- 172.21.15.43:0/1119243051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884037070 con 0x7f98940cd180 2024-04-26T20:32:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9892a00640 1 -- 172.21.15.43:0/1119243051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884032440 con 0x7f98940cd180 2024-04-26T20:32:04.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9892a00640 1 -- 172.21.15.43:0/1119243051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884032810 con 0x7f98940cd180 2024-04-26T20:32:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9899400640 1 -- 172.21.15.43:0/1119243051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98940cd180 msgr2=0x7f9894072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9899400640 1 --2- 172.21.15.43:0/1119243051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98940cd180 0x7f9894072620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f988402a2a0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9899400640 1 -- 172.21.15.43:0/1119243051 shutdown_connections 2024-04-26T20:32:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9899400640 1 --2- 172.21.15.43:0/1119243051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98940cd180 0x7f9894072620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.532+0000 7f9899400640 1 -- 172.21.15.43:0/1119243051 >> 172.21.15.43:0/1119243051 conn(0x7f9894068090 msgr2=0x7f9894068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.533+0000 7f9899400640 1 -- 172.21.15.43:0/1119243051 shutdown_connections 2024-04-26T20:32:04.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.533+0000 7f9899400640 1 -- 172.21.15.43:0/1119243051 wait complete. 2024-04-26T20:32:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.534+0000 7f9899400640 1 Processor -- start 2024-04-26T20:32:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.534+0000 7f9899400640 1 -- start start 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 0x7f9894142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 0x7f989415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f989415e0d0 con 0x7f989415dc50 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9894073da0 con 0x7f98940ce480 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.535+0000 7f9899400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f989406e1d0 con 0x7f98940cd180 2024-04-26T20:32:04.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43806/0 (socket says 172.21.15.43:43806) 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 -- 172.21.15.43:0/1929709564 learned_addr learned my addr 172.21.15.43:0/1929709564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9893400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 0x7f989415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9893e00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 0x7f9894142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 -- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 msgr2=0x7f9894142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 0x7f9894142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 -- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 msgr2=0x7f989415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 0x7f989415c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9898a00640 1 -- 172.21.15.43:0/1929709564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989415ca30 con 0x7f989415dc50 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9893e00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 0x7f9894142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.536+0000 7f9893400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 0x7f989415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:04.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f9898a00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c780 tx=0x7f988c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c004b60 con 0x7f989415dc50 2024-04-26T20:32:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988c004d00 con 0x7f989415dc50 2024-04-26T20:32:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00aa90 con 0x7f989415dc50 2024-04-26T20:32:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98941f5f60 con 0x7f989415dc50 2024-04-26T20:32:04.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.537+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98940b0820 con 0x7f989415dc50 2024-04-26T20:32:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.538+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9894069350 con 0x7f989415dc50 2024-04-26T20:32:04.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.539+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f988c00ac30 con 0x7f989415dc50 2024-04-26T20:32:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.540+0000 7f988be00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 0x7f987406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.540+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f988c08bfe0 con 0x7f989415dc50 2024-04-26T20:32:04.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.541+0000 7f9893e00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 0x7f987406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.541+0000 7f9893e00640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 0x7f987406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9884034000 tx=0x7f988402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.545+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f988c020070 con 0x7f989415dc50 2024-04-26T20:32:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.751+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98940d0670 con 0x7f989415dc50 2024-04-26T20:32:04.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.754+0000 7f988be00640 1 -- 172.21.15.43:0/1929709564 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16967 (secure 0 0 0) 0x7f989415ca30 con 0x7f989415dc50 2024-04-26T20:32:04.756 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:04.757 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:28:45.149793+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:32:04.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 msgr2=0x7f987406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 0x7f987406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9884034000 tx=0x7f988402e000 comp rx=0 tx=0).stop 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 msgr2=0x7f989413f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c780 tx=0x7f988c00cc40 comp rx=0 tx=0).stop 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 shutdown_connections 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f987406c8f0 0x7f987406edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f989415dc50 0x7f989413f8f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98940ce480 0x7f989415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 --2- 172.21.15.43:0/1929709564 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98940cd180 0x7f9894142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.757+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 >> 172.21.15.43:0/1929709564 conn(0x7f9894068090 msgr2=0x7f9894145ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.758+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 shutdown_connections 2024-04-26T20:32:04.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.758+0000 7f9899400640 1 -- 172.21.15.43:0/1929709564 wait complete. 2024-04-26T20:32:04.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:25:50.893409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:27:47.335104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:27:48.321573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:32:04.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:32:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.987+0000 7f5881e00640 1 Processor -- start 2024-04-26T20:32:04.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.988+0000 7f5881e00640 1 -- start start 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5881e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5881e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f587c06de70 0x7f587c162df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5881e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587c06e650 con 0x7f587c163330 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5881e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f587c0b6df0 con 0x7f587c06de70 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5881e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f587c06e370 con 0x7f587c0b44b0 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5873e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f587c06de70 0x7f587c162df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5873e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f587c06de70 0x7f587c162df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43818/0 (socket says 172.21.15.43:43818) 2024-04-26T20:32:04.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.989+0000 7f5873e00640 1 -- 172.21.15.43:0/3833974912 learned_addr learned my addr 172.21.15.43:0/3833974912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.990+0000 7f5873e00640 1 -- 172.21.15.43:0/3833974912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 msgr2=0x7f587c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.990+0000 7f587aa00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.990+0000 7f5873e00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c06d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.990+0000 7f5873e00640 1 -- 172.21.15.43:0/3833974912 >> v1:172.21.15.43:6789/0 conn(0x7f587c163330 legacy=0x7f587c15ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.990+0000 7f5873e00640 1 -- 172.21.15.43:0/3833974912 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0b67a0 con 0x7f587c06de70 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.991+0000 7f5873e00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c06de70 0x7f587c162df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab1d21f95f03c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.991+0000 7f587a000640 1 -- 172.21.15.43:0/3833974912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864037070 con 0x7f587c06de70 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.991+0000 7f587a000640 1 -- 172.21.15.43:0/3833974912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864032440 con 0x7f587c06de70 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.991+0000 7f587a000640 1 -- 172.21.15.43:0/3833974912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864032810 con 0x7f587c06de70 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 -- 172.21.15.43:0/3833974912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c06de70 msgr2=0x7f587c162df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c06de70 0x7f587c162df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).stop 2024-04-26T20:32:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 -- 172.21.15.43:0/3833974912 shutdown_connections 2024-04-26T20:32:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c06de70 0x7f587c162df0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 --2- 172.21.15.43:0/3833974912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 -- 172.21.15.43:0/3833974912 >> 172.21.15.43:0/3833974912 conn(0x7f587c0d2da0 msgr2=0x7f587c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 -- 172.21.15.43:0/3833974912 shutdown_connections 2024-04-26T20:32:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.992+0000 7f5881e00640 1 -- 172.21.15.43:0/3833974912 wait complete. 2024-04-26T20:32:04.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.994+0000 7f5881e00640 1 Processor -- start 2024-04-26T20:32:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.994+0000 7f5881e00640 1 -- start start 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 0x7f587c1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f587c15f120 con 0x7f587c163330 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0d23b0 con 0x7f587c06de70 2024-04-26T20:32:04.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.995+0000 7f5881e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f587c15f2a0 con 0x7f587c0b44b0 2024-04-26T20:32:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43820/0 (socket says 172.21.15.43:43820) 2024-04-26T20:32:04.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 -- 172.21.15.43:0/391477761 learned_addr learned my addr 172.21.15.43:0/391477761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587aa00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 0x7f587c1f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f5873e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 -- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 msgr2=0x7f587c0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 -- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 msgr2=0x7f587c1f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 0x7f587c1f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587b400640 1 -- 172.21.15.43:0/391477761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0ca510 con 0x7f587c163330 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f5873e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:04.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.996+0000 7f587aa00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 0x7f587c1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f587b400640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f587400c6b0 tx=0x7f587400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874003b70 con 0x7f587c163330 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5874003d30 con 0x7f587c163330 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874005310 con 0x7f587c163330 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c0cfba0 con 0x7f587c163330 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.997+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c0d0160 con 0x7f587c163330 2024-04-26T20:32:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.998+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c0c9820 con 0x7f587c163330 2024-04-26T20:32:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:04.999+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58740054b0 con 0x7f587c163330 2024-04-26T20:32:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.000+0000 7f5873400640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.000+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f587408bc30 con 0x7f587c163330 2024-04-26T20:32:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.001+0000 7f587aa00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.001+0000 7f587aa00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 0x7f585806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f586c008890 tx=0x7f586c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.005+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f587c0c9820 con 0x7f587c163330 2024-04-26T20:32:05.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.212+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f587c069a60 con 0x7f587c163330 2024-04-26T20:32:05.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.213+0000 7f5873400640 1 -- 172.21.15.43:0/391477761 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f5874059e10 con 0x7f587c163330 2024-04-26T20:32:05.214 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-26T20:32:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 msgr2=0x7f585806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 0x7f585806edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f586c008890 tx=0x7f586c008040 comp rx=0 tx=0).stop 2024-04-26T20:32:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 msgr2=0x7f587c1f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f587400c6b0 tx=0x7f587400cb70 comp rx=0 tx=0).stop 2024-04-26T20:32:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 shutdown_connections 2024-04-26T20:32:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f585806c8f0 0x7f585806edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f587c0b44b0 0x7f587c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f587c163330 0x7f587c1f8af0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 --2- 172.21.15.43:0/391477761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f587c06de70 0x7f587c1f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.216+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 >> 172.21.15.43:0/391477761 conn(0x7f587c0d2da0 msgr2=0x7f587c0b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.217+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 shutdown_connections 2024-04-26T20:32:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.217+0000 7f5881e00640 1 -- 172.21.15.43:0/391477761 wait complete. 2024-04-26T20:32:05.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:32:05.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.451+0000 7f1ff9e00640 1 Processor -- start 2024-04-26T20:32:05.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.451+0000 7f1ff9e00640 1 -- start start 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff406e6f0 0x7f1ff40721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff4163a50 con 0x7f1ff406db80 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff4163800 con 0x7f1ff406e6f0 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff4163d30 con 0x7f1ff4160d10 2024-04-26T20:32:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff406e6f0 0x7f1ff40721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff406e6f0 0x7f1ff40721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43828/0 (socket says 172.21.15.43:43828) 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.453+0000 7f1ff9400640 1 -- 172.21.15.43:0/4204160410 learned_addr learned my addr 172.21.15.43:0/4204160410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.454+0000 7f1ff8a00640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.454+0000 7f1ff9400640 1 -- 172.21.15.43:0/4204160410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 msgr2=0x7f1ff40b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.454+0000 7f1ff9400640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.454+0000 7f1ff9400640 1 -- 172.21.15.43:0/4204160410 >> v1:172.21.15.164:6789/0 conn(0x7f1ff406db80 legacy=0x7f1ff406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.454+0000 7f1ff9400640 1 -- 172.21.15.43:0/4204160410 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff4164010 con 0x7f1ff406e6f0 2024-04-26T20:32:05.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.455+0000 7f1ff9400640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40721e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e50 tx=0x7f1fec02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a884863e87dcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.455+0000 7f1ff3400640 1 -- 172.21.15.43:0/4204160410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec036070 con 0x7f1ff406e6f0 2024-04-26T20:32:05.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.455+0000 7f1ff3400640 1 -- 172.21.15.43:0/4204160410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec031600 con 0x7f1ff406e6f0 2024-04-26T20:32:05.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.455+0000 7f1ff3400640 1 -- 172.21.15.43:0/4204160410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0319d0 con 0x7f1ff406e6f0 2024-04-26T20:32:05.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 -- 172.21.15.43:0/4204160410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 msgr2=0x7f1ff40721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40721e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009e50 tx=0x7f1fec02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 -- 172.21.15.43:0/4204160410 shutdown_connections 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40721e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/4204160410 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 -- 172.21.15.43:0/4204160410 >> 172.21.15.43:0/4204160410 conn(0x7f1ff4068090 msgr2=0x7f1ff40684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 -- 172.21.15.43:0/4204160410 shutdown_connections 2024-04-26T20:32:05.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.456+0000 7f1ff9e00640 1 -- 172.21.15.43:0/4204160410 wait complete. 2024-04-26T20:32:05.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.458+0000 7f1ff9e00640 1 Processor -- start 2024-04-26T20:32:05.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.458+0000 7f1ff9e00640 1 -- start start 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 0x7f1ff41fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4164950 con 0x7f1ff406e6f0 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41610f0 con 0x7f1ff406db80 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff9e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff4164ad0 con 0x7f1ff4160d10 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.459+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43832/0 (socket says 172.21.15.43:43832) 2024-04-26T20:32:05.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 -- 172.21.15.43:0/290407873 learned_addr learned my addr 172.21.15.43:0/290407873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:05.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff8a00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 0x7f1ff41fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff9400640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 -- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 msgr2=0x7f1ff40b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 -- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 msgr2=0x7f1ff41fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 0x7f1ff41fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 -- 172.21.15.43:0/290407873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff41fc760 con 0x7f1ff406e6f0 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff8a00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 0x7f1ff41fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff9400640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.460+0000 7f1ff3e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c460 tx=0x7f1fe000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.461+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0016020 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.461+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0012870 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.461+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0012c60 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.461+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41e9f50 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.461+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41ea510 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.462+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff40506c0 con 0x7f1ff406e6f0 2024-04-26T20:32:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.463+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fe0019020 con 0x7f1ff406e6f0 2024-04-26T20:32:05.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.465+0000 7f1ff0c00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 0x7f1fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.465+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f1fe008bb10 con 0x7f1ff406e6f0 2024-04-26T20:32:05.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.465+0000 7f1ff8a00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 0x7f1fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.466+0000 7f1ff8a00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 0x7f1fd406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d1300 tx=0x7f1fe8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.469+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fe0020030 con 0x7f1ff406e6f0 2024-04-26T20:32:05.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.674+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ff40b57a0 con 0x7f1ff406e6f0 2024-04-26T20:32:05.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.676+0000 7f1ff0c00640 1 -- 172.21.15.43:0/290407873 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f1fe00182c0 con 0x7f1ff406e6f0 2024-04-26T20:32:05.677 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-26T20:32:05.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 msgr2=0x7f1fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 0x7f1fd406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40d1300 tx=0x7f1fe8008040 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 msgr2=0x7f1ff40adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c460 tx=0x7f1fe000c920 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 shutdown_connections 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f1fd406c8f0 0x7f1fd406edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1ff4160d10 0x7f1ff40b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff406e6f0 0x7f1ff40adef0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 --2- 172.21.15.43:0/290407873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1ff406db80 0x7f1ff41fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 >> 172.21.15.43:0/290407873 conn(0x7f1ff4068090 msgr2=0x7f1ff415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 shutdown_connections 2024-04-26T20:32:05.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.679+0000 7f1ff9e00640 1 -- 172.21.15.43:0/290407873 wait complete. 2024-04-26T20:32:05.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:32:05.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.915+0000 7f48f1000640 1 Processor -- start 2024-04-26T20:32:05.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.916+0000 7f48f1000640 1 -- start start 2024-04-26T20:32:05.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48f1000640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48ec15f180 0x7f48ec0d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48f1000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48ec0d1b00 0x7f48ec0d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48f1000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ec0d0490 con 0x7f48ec0d57d0 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48f1000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec0d1340 con 0x7f48ec0d1b00 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48f1000640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec0d3ee0 con 0x7f48ec15f180 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48ec0d1b00 0x7f48ec0d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48ec0d1b00 0x7f48ec0d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51628/0 (socket says 172.21.15.43:51628) 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 -- 172.21.15.43:0/4094153062 learned_addr learned my addr 172.21.15.43:0/4094153062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 -- 172.21.15.43:0/4094153062 >> v2:172.21.15.164:3300/0 conn(0x7f48ec15f180 msgr2=0x7f48ec0d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:05.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 --2- 172.21.15.43:0/4094153062 >> v2:172.21.15.164:3300/0 conn(0x7f48ec15f180 0x7f48ec0d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.918+0000 7f48ebe00640 1 -- 172.21.15.43:0/4094153062 >> v1:172.21.15.164:6789/0 conn(0x7f48ec0d57d0 legacy=0x7f48ec0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.919+0000 7f48ebe00640 1 -- 172.21.15.43:0/4094153062 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec0d0730 con 0x7f48ec0d1b00 2024-04-26T20:32:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.920+0000 7f48ebe00640 1 --2- 172.21.15.43:0/4094153062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec0d39a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48ec08d450 tx=0x7f48e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bae7d70aacd3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.920+0000 7f48eaa00640 1 -- 172.21.15.43:0/4094153062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4036070 con 0x7f48ec0d1b00 2024-04-26T20:32:05.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.920+0000 7f48eaa00640 1 -- 172.21.15.43:0/4094153062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4031600 con 0x7f48ec0d1b00 2024-04-26T20:32:05.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.920+0000 7f48eaa00640 1 -- 172.21.15.43:0/4094153062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40319d0 con 0x7f48ec0d1b00 2024-04-26T20:32:05.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 -- 172.21.15.43:0/4094153062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 msgr2=0x7f48ec0d39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 --2- 172.21.15.43:0/4094153062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec0d39a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f48ec08d450 tx=0x7f48e402a660 comp rx=0 tx=0).stop 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 -- 172.21.15.43:0/4094153062 shutdown_connections 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 --2- 172.21.15.43:0/4094153062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec0d39a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 --2- 172.21.15.43:0/4094153062 >> v2:172.21.15.164:3300/0 conn(0x7f48ec15f180 0x7f48ec0d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 -- 172.21.15.43:0/4094153062 >> 172.21.15.43:0/4094153062 conn(0x7f48ec0d4bb0 msgr2=0x7f48ec0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 -- 172.21.15.43:0/4094153062 shutdown_connections 2024-04-26T20:32:05.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.921+0000 7f48f1000640 1 -- 172.21.15.43:0/4094153062 wait complete. 2024-04-26T20:32:05.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.923+0000 7f48f1000640 1 Processor -- start 2024-04-26T20:32:05.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.923+0000 7f48f1000640 1 -- start start 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 0x7f48ec0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 0x7f48ec0ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0d0cf0 con 0x7f48ec0d1b00 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0d1f30 con 0x7f48ec15f180 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48f1000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0d0fd0 con 0x7f48ec0d57d0 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51644/0 (socket says 172.21.15.43:51644) 2024-04-26T20:32:05.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48eb400640 1 -- 172.21.15.43:0/2612700128 learned_addr learned my addr 172.21.15.43:0/2612700128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48e3e00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 0x7f48ec0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.925+0000 7f48ebe00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 0x7f48ec0ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 -- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 msgr2=0x7f48ec0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 0x7f48ec0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 -- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 msgr2=0x7f48ec0ba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 0x7f48ec0ba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 -- 172.21.15.43:0/2612700128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec0b9140 con 0x7f48ec0d1b00 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48e3e00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 0x7f48ec0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48ebe00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 0x7f48ec0ba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48eb400640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b4a0 tx=0x7f48dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc014070 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc010610 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc010a00 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec0b9420 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.926+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec0b99e0 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.928+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0506c0 con 0x7f48ec0d1b00 2024-04-26T20:32:05.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.929+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48dc017070 con 0x7f48ec0d1b00 2024-04-26T20:32:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.930+0000 7f48e3400640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.930+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f48dc08b7f0 con 0x7f48ec0d1b00 2024-04-26T20:32:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.930+0000 7f48e3e00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:05.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.931+0000 7f48e3e00640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 0x7f48c806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48ec161a90 tx=0x7f48d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:05.935+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48dc0599d0 con 0x7f48ec0d1b00 2024-04-26T20:32:06.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.141+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48ec15e6e0 con 0x7f48ec0d1b00 2024-04-26T20:32:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.142+0000 7f48e3400640 1 -- 172.21.15.43:0/2612700128 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f48dc0592d0 con 0x7f48ec0d1b00 2024-04-26T20:32:06.143 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-26T20:32:06.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 msgr2=0x7f48c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 0x7f48c806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48ec161a90 tx=0x7f48d4008040 comp rx=0 tx=0).stop 2024-04-26T20:32:06.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 msgr2=0x7f48ec15add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00b4a0 tx=0x7f48dc00b960 comp rx=0 tx=0).stop 2024-04-26T20:32:06.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 shutdown_connections 2024-04-26T20:32:06.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f48c806c8f0 0x7f48c806edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48ec0d57d0 0x7f48ec0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ec15f180 0x7f48ec0ba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 --2- 172.21.15.43:0/2612700128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48ec0d1b00 0x7f48ec15add0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.144+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 >> 172.21.15.43:0/2612700128 conn(0x7f48ec0d4bb0 msgr2=0x7f48ec0cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.145+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 shutdown_connections 2024-04-26T20:32:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.145+0000 7f48f1000640 1 -- 172.21.15.43:0/2612700128 wait complete. 2024-04-26T20:32:06.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:32:06.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.378+0000 7fcedaa00640 1 Processor -- start 2024-04-26T20:32:06.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.378+0000 7fcedaa00640 1 -- start start 2024-04-26T20:32:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.379+0000 7fcedaa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fced40d36b0 0x7fced40d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.379+0000 7fcedaa00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced40d2160 con 0x7fced40cfb20 2024-04-26T20:32:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fcedaa00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced40cd280 con 0x7fced40b8570 2024-04-26T20:32:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fcedaa00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fced40cf050 con 0x7fced40d36b0 2024-04-26T20:32:06.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fced3400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fced40cfb20 0x7fced415e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51898/0 (socket says 172.21.15.43:51898) 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fced3400640 1 -- 172.21.15.43:0/496486691 learned_addr learned my addr 172.21.15.43:0/496486691 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fced8a00640 1 --2- 172.21.15.43:0/496486691 >> v2:172.21.15.164:3300/0 conn(0x7fced40d36b0 0x7fced40d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.380+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265558943 0 0) 0x7fced40d2160 con 0x7fced40cfb20 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb8003520 con 0x7fced40cfb20 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1791959405 0 0) 0x7fceb8003520 con 0x7fced40cfb20 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 >> v2:172.21.15.164:3300/0 conn(0x7fced40d36b0 msgr2=0x7fced40d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 --2- 172.21.15.43:0/496486691 >> v2:172.21.15.164:3300/0 conn(0x7fced40d36b0 0x7fced40d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 >> v1:172.21.15.164:6789/0 conn(0x7fced40b8570 legacy=0x7fced40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced40cd560 con 0x7fced40cfb20 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fcec4001670 con 0x7fced40cfb20 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 >> v1:172.21.15.43:6789/0 conn(0x7fced40cfb20 legacy=0x7fced415e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fceb8003a00 0x7fceb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fceb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8005210 0x7fceb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fced40cf050 con 0x7fceb8004800 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fceb800c160 con 0x7fceb8005210 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fceb800c360 con 0x7fceb8003a00 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.381+0000 7fced3400640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fceb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced3400640 1 -- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fceb8003a00 msgr2=0x7fceb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced3400640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fceb8003a00 0x7fceb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced3400640 1 -- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8005210 msgr2=0x7fceb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced8a00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8005210 0x7fceb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced3400640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8005210 0x7fceb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced3400640 1 -- 172.21.15.43:0/496486691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb8003520 con 0x7fceb8004800 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced8a00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fceb8005210 0x7fceb800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:06.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.382+0000 7fced8a00640 1 -- 172.21.15.43:0/496486691 reap_dead start 2024-04-26T20:32:06.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.383+0000 7fced3400640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fceb8004c80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007fd0 tx=0x7fceb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398242c7c7d4641a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.383+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4007c20 con 0x7fceb8004800 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.383+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4005af0 con 0x7fceb8004800 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.383+0000 7fced2a00640 1 -- 172.21.15.43:0/496486691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4007070 con 0x7fceb8004800 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 -- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 msgr2=0x7fceb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fceb8004c80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007fd0 tx=0x7fceb8005830 comp rx=0 tx=0).stop 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 -- 172.21.15.43:0/496486691 shutdown_connections 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 --2- 172.21.15.43:0/496486691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fceb8004c80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 -- 172.21.15.43:0/496486691 >> 172.21.15.43:0/496486691 conn(0x7fced40d2b90 msgr2=0x7fced40d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 -- 172.21.15.43:0/496486691 shutdown_connections 2024-04-26T20:32:06.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.384+0000 7fcedaa00640 1 -- 172.21.15.43:0/496486691 wait complete. 2024-04-26T20:32:06.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.386+0000 7fcedaa00640 1 Processor -- start 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.386+0000 7fcedaa00640 1 -- start start 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415d510 0x7fced40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 0x7fced415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fced40d1df0 con 0x7fceb8004800 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fced40cdb20 con 0x7fced415d510 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fcedaa00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fced40b59e0 con 0x7fced415d990 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fced3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fced3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51666/0 (socket says 172.21.15.43:51666) 2024-04-26T20:32:06.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.387+0000 7fced3e00640 1 -- 172.21.15.43:0/3673152306 learned_addr learned my addr 172.21.15.43:0/3673152306 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3400640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415d510 0x7fced40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 -- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 msgr2=0x7fced415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced8a00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 0x7fced415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 0x7fced415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 -- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415d510 msgr2=0x7fced40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415d510 0x7fced40cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 -- 172.21.15.43:0/3673152306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced4151420 con 0x7fceb8004800 2024-04-26T20:32:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced8a00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 0x7fced415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.388+0000 7fced3e00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c530 tx=0x7fcebc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.389+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0052d0 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.389+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcebc005470 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.389+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc012850 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.389+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced40cb1e0 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.389+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced40cb7a0 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.390+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced4069350 con 0x7fceb8004800 2024-04-26T20:32:06.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.391+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcebc0129f0 con 0x7fceb8004800 2024-04-26T20:32:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.393+0000 7fcecbe00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 0x7fceb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.393+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fcebc08ba00 con 0x7fceb8004800 2024-04-26T20:32:06.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.393+0000 7fced3400640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 0x7fceb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.394+0000 7fced3400640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 0x7fceb006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcec4032000 tx=0x7fcec4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.397+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcebc059c70 con 0x7fceb8004800 2024-04-26T20:32:06.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.633+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fced40cee10 con 0x7fceb8004800 2024-04-26T20:32:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.634+0000 7fcecbe00640 1 -- 172.21.15.43:0/3673152306 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fcebc059570 con 0x7fceb8004800 2024-04-26T20:32:06.636 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:06.636 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:32:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 msgr2=0x7fceb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 0x7fceb006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcec4032000 tx=0x7fcec4009030 comp rx=0 tx=0).stop 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 msgr2=0x7fced4150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c530 tx=0x7fcebc00c9f0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 shutdown_connections 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fceb006c8f0 0x7fceb006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fced415d990 0x7fced415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fced415d510 0x7fced40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 --2- 172.21.15.43:0/3673152306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb8004800 0x7fced4150ee0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 >> 172.21.15.43:0/3673152306 conn(0x7fced40d2b90 msgr2=0x7fced4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 shutdown_connections 2024-04-26T20:32:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.637+0000 7fcedaa00640 1 -- 172.21.15.43:0/3673152306 wait complete. 2024-04-26T20:32:06.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:32:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.857+0000 7f0dd0a00640 1 Processor -- start 2024-04-26T20:32:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.857+0000 7f0dd0a00640 1 -- start start 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.858+0000 7f0dd0a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dcc0cd180 0x7f0dcc072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.858+0000 7f0dd0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc0ce480 0x7f0dcc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dd0a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dcc0cd560 con 0x7f0dcc15dc50 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dd0a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0dcc072d00 con 0x7f0dcc0cd180 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dd0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc072fe0 con 0x7f0dcc0ce480 2024-04-26T20:32:06.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dcc0cd180 0x7f0dcc072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dcc0cd180 0x7f0dcc072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51680/0 (socket says 172.21.15.43:51680) 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 -- 172.21.15.43:0/1583844614 learned_addr learned my addr 172.21.15.43:0/1583844614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 -- 172.21.15.43:0/1583844614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc0ce480 msgr2=0x7f0dcc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 --2- 172.21.15.43:0/1583844614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc0ce480 0x7f0dcc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 -- 172.21.15.43:0/1583844614 >> v1:172.21.15.164:6789/0 conn(0x7f0dcc15dc50 legacy=0x7f0dcc0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:06.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.859+0000 7f0dca800640 1 -- 172.21.15.43:0/1583844614 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc0732c0 con 0x7f0dcc0cd180 2024-04-26T20:32:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.860+0000 7f0dca800640 1 --2- 172.21.15.43:0/1583844614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc072620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009e50 tx=0x7f0dbc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed5da13786ccb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dc9400640 1 -- 172.21.15.43:0/1583844614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc036070 con 0x7f0dcc0cd180 2024-04-26T20:32:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dc9400640 1 -- 172.21.15.43:0/1583844614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc031600 con 0x7f0dcc0cd180 2024-04-26T20:32:06.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dc9400640 1 -- 172.21.15.43:0/1583844614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0319d0 con 0x7f0dcc0cd180 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 -- 172.21.15.43:0/1583844614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0cd180 msgr2=0x7f0dcc072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/1583844614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc072620 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009e50 tx=0x7f0dbc02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 -- 172.21.15.43:0/1583844614 shutdown_connections 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/1583844614 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc0ce480 0x7f0dcc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/1583844614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc072620 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 -- 172.21.15.43:0/1583844614 >> 172.21.15.43:0/1583844614 conn(0x7f0dcc068090 msgr2=0x7f0dcc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.861+0000 7f0dd0a00640 1 -- 172.21.15.43:0/1583844614 shutdown_connections 2024-04-26T20:32:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.862+0000 7f0dd0a00640 1 -- 172.21.15.43:0/1583844614 wait complete. 2024-04-26T20:32:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.863+0000 7f0dd0a00640 1 Processor -- start 2024-04-26T20:32:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.863+0000 7f0dd0a00640 1 -- start start 2024-04-26T20:32:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.864+0000 7f0dd0a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc1f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.864+0000 7f0dd0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dd0a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc15dc50 0x7f0dcc143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dd0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc15e0d0 con 0x7f0dcc0ce480 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dd0a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc073da0 con 0x7f0dcc0cd180 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dd0a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc06e000 con 0x7f0dcc15dc50 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51690/0 (socket says 172.21.15.43:51690) 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 -- 172.21.15.43:0/622477187 learned_addr learned my addr 172.21.15.43:0/622477187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 -- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc15dc50 msgr2=0x7f0dcc143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc15dc50 0x7f0dcc143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 -- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dcc0cd180 msgr2=0x7f0dcc1f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc1f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.865+0000 7f0dc9e00640 1 -- 172.21.15.43:0/622477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc15cb30 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dc9e00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b410 tx=0x7f0db400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4015020 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db4011830 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011c20 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc142a00 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.866+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc143260 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.867+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc069350 con 0x7f0dcc0ce480 2024-04-26T20:32:06.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.868+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0db4018020 con 0x7f0dcc0ce480 2024-04-26T20:32:06.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.870+0000 7f0dc2a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 0x7f0da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:06.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.870+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f0db408bd50 con 0x7f0dcc0ce480 2024-04-26T20:32:06.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.870+0000 7f0dca800640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 0x7f0da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:06.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.871+0000 7f0dca800640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 0x7f0da806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc033000 tx=0x7f0dbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:06.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:06.874+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0db401f020 con 0x7f0dcc0ce480 2024-04-26T20:32:07.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.085+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dcc0d0670 con 0x7f0dcc0ce480 2024-04-26T20:32:07.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.088+0000 7f0dc2a00640 1 -- 172.21.15.43:0/622477187 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16967 (secure 0 0 0) 0x7f0dcc15cb30 con 0x7f0dcc0ce480 2024-04-26T20:32:07.091 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:28:45.149793+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:32:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 msgr2=0x7f0da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 0x7f0da806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc033000 tx=0x7f0dbc02e000 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 msgr2=0x7f0dcc15c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b410 tx=0x7f0db400b8d0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 shutdown_connections 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f0da806c8f0 0x7f0da806edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0dcc15dc50 0x7f0dcc143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dcc0ce480 0x7f0dcc15c5f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 --2- 172.21.15.43:0/622477187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dcc0cd180 0x7f0dcc1f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.091+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 >> 172.21.15.43:0/622477187 conn(0x7f0dcc068090 msgr2=0x7f0dcc0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.092+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 shutdown_connections 2024-04-26T20:32:07.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.092+0000 7f0dd0a00640 1 -- 172.21.15.43:0/622477187 wait complete. 2024-04-26T20:32:07.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:32:07.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:32:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.328+0000 7ff832600640 1 Processor -- start 2024-04-26T20:32:07.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.328+0000 7ff832600640 1 -- start start 2024-04-26T20:32:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff832600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 0x7ff82c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff832600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff832600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82c146ba0 con 0x7ff82c1467c0 2024-04-26T20:32:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff832600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c073dd0 con 0x7ff82c072b70 2024-04-26T20:32:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff832600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0731c0 con 0x7ff82c0d1060 2024-04-26T20:32:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff82be00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.330+0000 7ff82aa00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 0x7ff82c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:35290/0 (socket says 172.21.15.43:35290) 2024-04-26T20:32:07.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 -- 172.21.15.43:0/193145888 learned_addr learned my addr 172.21.15.43:0/193145888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82b400640 1 --1- 172.21.15.43:0/193145888 >> v1:172.21.15.164:6789/0 conn(0x7ff82c1467c0 0x7ff82c0b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:53730/0 (socket says 172.21.15.43:53730) 2024-04-26T20:32:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 -- 172.21.15.43:0/193145888 >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 msgr2=0x7ff82c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 --2- 172.21.15.43:0/193145888 >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 0x7ff82c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 -- 172.21.15.43:0/193145888 >> v1:172.21.15.164:6789/0 conn(0x7ff82c1467c0 legacy=0x7ff82c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82be00640 1 -- 172.21.15.43:0/193145888 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0734a0 con 0x7ff82c0d1060 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.331+0000 7ff82aa00640 1 --2- 172.21.15.43:0/193145888 >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 0x7ff82c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.333+0000 7ff82be00640 1 --2- 172.21.15.43:0/193145888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff82402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f44678f32c471bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.333+0000 7ff82a000640 1 -- 172.21.15.43:0/193145888 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824037070 con 0x7ff82c0d1060 2024-04-26T20:32:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.333+0000 7ff82a000640 1 -- 172.21.15.43:0/193145888 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824032440 con 0x7ff82c0d1060 2024-04-26T20:32:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.333+0000 7ff82a000640 1 -- 172.21.15.43:0/193145888 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824032810 con 0x7ff82c0d1060 2024-04-26T20:32:07.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 -- 172.21.15.43:0/193145888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 msgr2=0x7ff82c06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 --2- 172.21.15.43:0/193145888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff82402a250 comp rx=0 tx=0).stop 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 -- 172.21.15.43:0/193145888 shutdown_connections 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 --2- 172.21.15.43:0/193145888 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c06dff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 --2- 172.21.15.43:0/193145888 >> v2:172.21.15.164:3300/0 conn(0x7ff82c072b70 0x7ff82c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 -- 172.21.15.43:0/193145888 >> 172.21.15.43:0/193145888 conn(0x7ff82c0b2900 msgr2=0x7ff82c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 -- 172.21.15.43:0/193145888 shutdown_connections 2024-04-26T20:32:07.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.334+0000 7ff832600640 1 -- 172.21.15.43:0/193145888 wait complete. 2024-04-26T20:32:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.336+0000 7ff832600640 1 Processor -- start 2024-04-26T20:32:07.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.336+0000 7ff832600640 1 -- start start 2024-04-26T20:32:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c1467c0 0x7ff82c1404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c06e530 con 0x7ff82c072b70 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c073a60 con 0x7ff82c1467c0 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff832600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c06e6b0 con 0x7ff82c0d1060 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff82b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff82b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51694/0 (socket says 172.21.15.43:51694) 2024-04-26T20:32:07.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.337+0000 7ff82b400640 1 -- 172.21.15.43:0/2528190584 learned_addr learned my addr 172.21.15.43:0/2528190584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82be00640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c1467c0 0x7ff82c1404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82aa00640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 -- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 msgr2=0x7ff82c13ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c13ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 -- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c1467c0 msgr2=0x7ff82c1404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c1467c0 0x7ff82c1404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 -- 172.21.15.43:0/2528190584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c140b50 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff82b400640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c3f0 tx=0x7ff81c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0074e0 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c007680 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c140e30 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004a00 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.338+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c1413f0 con 0x7ff82c072b70 2024-04-26T20:32:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.340+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c069350 con 0x7ff82c072b70 2024-04-26T20:32:07.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.341+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff81c01a020 con 0x7ff82c072b70 2024-04-26T20:32:07.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.342+0000 7ff823400640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.342+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7ff81c08b4f0 con 0x7ff82c072b70 2024-04-26T20:32:07.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.342+0000 7ff82aa00640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.343+0000 7ff82aa00640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 0x7ff80806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1e5520 tx=0x7ff81400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.346+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff81c010300 con 0x7ff82c072b70 2024-04-26T20:32:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.593+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff82c0b1cb0 con 0x7ff82c072b70 2024-04-26T20:32:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.595+0000 7ff823400640 1 -- 172.21.15.43:0/2528190584 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7ff81c059760 con 0x7ff82c072b70 2024-04-26T20:32:07.596 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:07.596 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:32:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:32:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.597+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 msgr2=0x7ff80806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.597+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 0x7ff80806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1e5520 tx=0x7ff81400b040 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.597+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 msgr2=0x7ff82c1427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.597+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c3f0 tx=0x7ff81c00c8b0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 shutdown_connections 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff82c0d1060 0x7ff82c13ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c1467c0 0x7ff82c1404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 --2- 172.21.15.43:0/2528190584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82c072b70 0x7ff82c1427c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 >> 172.21.15.43:0/2528190584 conn(0x7ff82c0b2900 msgr2=0x7ff82c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 shutdown_connections 2024-04-26T20:32:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.598+0000 7ff832600640 1 -- 172.21.15.43:0/2528190584 wait complete. 2024-04-26T20:32:07.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:32:07.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.839+0000 7fbc6e400640 1 Processor -- start 2024-04-26T20:32:07.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.839+0000 7fbc6e400640 1 -- start start 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6e400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc680d3810 0x7fbc680b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6e400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc680d0480 con 0x7fbc680d1a00 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6e400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbc680d0ce0 con 0x7fbc680d3810 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbc680d0720 con 0x7fbc680d1620 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6da00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc680d3810 0x7fbc680b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6da00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc680d3810 0x7fbc680b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51708/0 (socket says 172.21.15.43:51708) 2024-04-26T20:32:07.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.840+0000 7fbc6da00640 1 -- 172.21.15.43:0/6492698 learned_addr learned my addr 172.21.15.43:0/6492698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.841+0000 7fbc6d000640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.841+0000 7fbc6da00640 1 -- 172.21.15.43:0/6492698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 msgr2=0x7fbc680d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.841+0000 7fbc6da00640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.841+0000 7fbc6da00640 1 -- 172.21.15.43:0/6492698 >> v1:172.21.15.43:6789/0 conn(0x7fbc680d1a00 legacy=0x7fbc680b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.841+0000 7fbc6da00640 1 -- 172.21.15.43:0/6492698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc680d0a00 con 0x7fbc680d3810 2024-04-26T20:32:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.842+0000 7fbc6da00640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d3810 0x7fbc680b5280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009e50 tx=0x7fbc6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582ef628858b4242 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.842+0000 7fbc67400640 1 -- 172.21.15.43:0/6492698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60036070 con 0x7fbc680d3810 2024-04-26T20:32:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc67400640 1 -- 172.21.15.43:0/6492698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc60031600 con 0x7fbc680d3810 2024-04-26T20:32:07.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc67400640 1 -- 172.21.15.43:0/6492698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600319d0 con 0x7fbc680d3810 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 -- 172.21.15.43:0/6492698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d3810 msgr2=0x7fbc680b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d3810 0x7fbc680b5280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009e50 tx=0x7fbc6002a680 comp rx=0 tx=0).stop 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 -- 172.21.15.43:0/6492698 shutdown_connections 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d3810 0x7fbc680b5280 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 --2- 172.21.15.43:0/6492698 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 -- 172.21.15.43:0/6492698 >> 172.21.15.43:0/6492698 conn(0x7fbc680d4c30 msgr2=0x7fbc680d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 -- 172.21.15.43:0/6492698 shutdown_connections 2024-04-26T20:32:07.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.843+0000 7fbc6e400640 1 -- 172.21.15.43:0/6492698 wait complete. 2024-04-26T20:32:07.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.845+0000 7fbc6e400640 1 Processor -- start 2024-04-26T20:32:07.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.845+0000 7fbc6e400640 1 -- start start 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 0x7fbc680b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc680b57c0 con 0x7fbc680d1a00 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc680d1e60 con 0x7fbc680d3810 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc6e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbc680b5940 con 0x7fbc680d1620 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51720/0 (socket says 172.21.15.43:51720) 2024-04-26T20:32:07.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.846+0000 7fbc67e00640 1 -- 172.21.15.43:0/1402294817 learned_addr learned my addr 172.21.15.43:0/1402294817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc6d000640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc6da00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 0x7fbc680b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 -- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 msgr2=0x7fbc680ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 -- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 msgr2=0x7fbc680b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 0x7fbc680b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 -- 172.21.15.43:0/1402294817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc680cbda0 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc6da00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 0x7fbc680b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.847+0000 7fbc67e00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c400 tx=0x7fbc5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.848+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580052d0 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.848+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc58005490 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.848+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc680b8780 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.848+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58007260 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.848+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc680b8d40 con 0x7fbc680d1a00 2024-04-26T20:32:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.849+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc68069350 con 0x7fbc680d1a00 2024-04-26T20:32:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.850+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc58007400 con 0x7fbc680d1a00 2024-04-26T20:32:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.851+0000 7fbc64c00640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 0x7fbc4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.851+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fbc5808b2a0 con 0x7fbc680d1a00 2024-04-26T20:32:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.851+0000 7fbc6d000640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 0x7fbc4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:07.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.852+0000 7fbc6d000640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 0x7fbc4c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c006fd0 tx=0x7fbc5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:07.856+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc58059510 con 0x7fbc680d1a00 2024-04-26T20:32:08.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.101+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc680cb3a0 con 0x7fbc680d1a00 2024-04-26T20:32:08.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.103+0000 7fbc64c00640 1 -- 172.21.15.43:0/1402294817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5181 (secure 0 0 0) 0x7fbc58058e10 con 0x7fbc680d1a00 2024-04-26T20:32:08.105 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:08.105 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:27:49.342228+0000","modified":"2024-04-26T20:27:52.451048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/37476793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":37476793},{"type":"v1","addr":"172.21.15.43:6837","nonce":37476793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-26T20:32:08.105 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.105+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 msgr2=0x7fbc4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.105+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 0x7fbc4c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c006fd0 tx=0x7fbc5c008040 comp rx=0 tx=0).stop 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.105+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 msgr2=0x7fbc680ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.105+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c400 tx=0x7fbc5800c8c0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 shutdown_connections 2024-04-26T20:32:08.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbc4c06c8f0 0x7fbc4c06edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc680d3810 0x7fbc680b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc680d1a00 0x7fbc680ce630 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 --2- 172.21.15.43:0/1402294817 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbc680d1620 0x7fbc680ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 >> 172.21.15.43:0/1402294817 conn(0x7fbc680d4c30 msgr2=0x7fbc680b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 shutdown_connections 2024-04-26T20:32:08.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.106+0000 7fbc6e400640 1 -- 172.21.15.43:0/1402294817 wait complete. 2024-04-26T20:32:08.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:32:08.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.339+0000 7fcbe8a00640 1 Processor -- start 2024-04-26T20:32:08.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.339+0000 7fcbe8a00640 1 -- start start 2024-04-26T20:32:08.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.340+0000 7fcbe8a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcbe40d5750 0x7fcbe40d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe8a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe40d0480 con 0x7fcbe40d1620 2024-04-26T20:32:08.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe8a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe40d0720 con 0x7fcbe40b5b80 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe8a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe40d5be0 con 0x7fcbe40d5750 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe3200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcbe40d5750 0x7fcbe40d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe3200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcbe40d5750 0x7fcbe40d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51734/0 (socket says 172.21.15.43:51734) 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe3200640 1 -- 172.21.15.43:0/1246852367 learned_addr learned my addr 172.21.15.43:0/1246852367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.341+0000 7fcbe3200640 1 -- 172.21.15.43:0/1246852367 >> v1:172.21.15.164:6789/0 conn(0x7fcbe40b5b80 legacy=0x7fcbe40d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.342+0000 7fcbe3200640 1 -- 172.21.15.43:0/1246852367 >> v1:172.21.15.43:6789/0 conn(0x7fcbe40d1620 legacy=0x7fcbe40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.342+0000 7fcbe3200640 1 -- 172.21.15.43:0/1246852367 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe40d0a00 con 0x7fcbe40d5750 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.342+0000 7fcbe1400640 1 -- 172.21.15.43:0/1246852367 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932697831 0 0) 0x7fcbe40d0480 con 0x7fcbe40d1620 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.343+0000 7fcbe3200640 1 --2- 172.21.15.43:0/1246852367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40d2f10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009870 tx=0x7fcbdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb48870ee051903 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.343+0000 7fcbe1400640 1 -- 172.21.15.43:0/1246852367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc037070 con 0x7fcbe40d5750 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.343+0000 7fcbe1400640 1 -- 172.21.15.43:0/1246852367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc032440 con 0x7fcbe40d5750 2024-04-26T20:32:08.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.343+0000 7fcbe1400640 1 -- 172.21.15.43:0/1246852367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc032810 con 0x7fcbe40d5750 2024-04-26T20:32:08.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1246852367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 msgr2=0x7fcbe40d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1246852367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40d2f10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009870 tx=0x7fcbdc02a270 comp rx=0 tx=0).stop 2024-04-26T20:32:08.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1246852367 shutdown_connections 2024-04-26T20:32:08.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1246852367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40d2f10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1246852367 >> 172.21.15.43:0/1246852367 conn(0x7fcbe40d4c30 msgr2=0x7fcbe40d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:08.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1246852367 shutdown_connections 2024-04-26T20:32:08.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.344+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1246852367 wait complete. 2024-04-26T20:32:08.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.346+0000 7fcbe8a00640 1 Processor -- start 2024-04-26T20:32:08.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.346+0000 7fcbe8a00640 1 -- start start 2024-04-26T20:32:08.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbe40b5b80 0x7fcbe415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 0x7fcbe41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe40d3450 con 0x7fcbe40d5750 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe40d0ce0 con 0x7fcbe40b5b80 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe8a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe40d35d0 con 0x7fcbe40d1620 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51748/0 (socket says 172.21.15.43:51748) 2024-04-26T20:32:08.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe3200640 1 -- 172.21.15.43:0/1591434619 learned_addr learned my addr 172.21.15.43:0/1591434619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe1e00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 0x7fcbe41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.347+0000 7fcbe2800640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbe40b5b80 0x7fcbe415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 -- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 msgr2=0x7fcbe41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 0x7fcbe41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 -- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbe40b5b80 msgr2=0x7fcbe415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbe40b5b80 0x7fcbe415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 -- 172.21.15.43:0/1591434619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe40c8590 con 0x7fcbe40d5750 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe1e00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 0x7fcbe41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe3200640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc02a2b0 tx=0x7fcbdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc041020 con 0x7fcbe40d5750 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc030990 con 0x7fcbe40d5750 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc037040 con 0x7fcbe40d5750 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe40b3090 con 0x7fcbe40d5750 2024-04-26T20:32:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.348+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe415beb0 con 0x7fcbe40d5750 2024-04-26T20:32:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.350+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe4069350 con 0x7fcbe40d5750 2024-04-26T20:32:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.351+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbdc044020 con 0x7fcbe40d5750 2024-04-26T20:32:08.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.352+0000 7fcbdaa00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 0x7fcbc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.352+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fcbdc0ac6a0 con 0x7fcbe40d5750 2024-04-26T20:32:08.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.352+0000 7fcbe2800640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 0x7fcbc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.353+0000 7fcbe2800640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 0x7fcbc006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40012e0 tx=0x7fcbd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.356+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbdc04b030 con 0x7fcbe40d5750 2024-04-26T20:32:08.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.567+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbe406ca90 con 0x7fcbe40d5750 2024-04-26T20:32:08.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.569+0000 7fcbdaa00640 1 -- 172.21.15.43:0/1591434619 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16967 (secure 0 0 0) 0x7fcbdc07a920 con 0x7fcbe40d5750 2024-04-26T20:32:08.571 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:08.572 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:28:45.149793+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:27:47.335104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:27:48.321573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 msgr2=0x7fcbc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 0x7fcbc006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40012e0 tx=0x7fcbd4009040 comp rx=0 tx=0).stop 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 msgr2=0x7fcbe40b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc02a2b0 tx=0x7fcbdc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 shutdown_connections 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fcbc006c8f0 0x7fcbc006edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbe40d1620 0x7fcbe41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbe40d5750 0x7fcbe40b2b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 --2- 172.21.15.43:0/1591434619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbe40b5b80 0x7fcbe415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.572+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 >> 172.21.15.43:0/1591434619 conn(0x7fcbe40d4c30 msgr2=0x7fcbe40d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.573+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 shutdown_connections 2024-04-26T20:32:08.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.573+0000 7fcbe8a00640 1 -- 172.21.15.43:0/1591434619 wait complete. 2024-04-26T20:32:08.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:32:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.809+0000 7f3608c00640 1 Processor -- start 2024-04-26T20:32:08.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.809+0000 7f3608c00640 1 -- start start 2024-04-26T20:32:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.810+0000 7f3608c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3604160f40 0x7f360415c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.810+0000 7f3608c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36040cd560 con 0x7f36040d36e0 2024-04-26T20:32:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.810+0000 7f3608c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3604070d60 con 0x7f3604071870 2024-04-26T20:32:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.810+0000 7f3608c00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f36040cd7e0 con 0x7f3604160f40 2024-04-26T20:32:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3602a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36040d36e0 0x7f3604160830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51936/0 (socket says 172.21.15.43:51936) 2024-04-26T20:32:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3602a00640 1 -- 172.21.15.43:0/4268747208 learned_addr learned my addr 172.21.15.43:0/4268747208 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3603e00640 1 --2- 172.21.15.43:0/4268747208 >> v2:172.21.15.43:3300/0 conn(0x7f3604160f40 0x7f360415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3603e00640 1 -- 172.21.15.43:0/4268747208 >> v1:172.21.15.164:6789/0 conn(0x7f3604071870 legacy=0x7f36040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3603e00640 1 -- 172.21.15.43:0/4268747208 >> v1:172.21.15.43:6789/0 conn(0x7f36040d36e0 legacy=0x7f3604160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.811+0000 7f3603e00640 1 -- 172.21.15.43:0/4268747208 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36040cdac0 con 0x7f3604160f40 2024-04-26T20:32:08.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.812+0000 7f3603e00640 1 --2- 172.21.15.43:0/4268747208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604160f40 0x7f360415c610 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0097a0 tx=0x7f35fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f8af76cfae0ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.813+0000 7f3602000640 1 -- 172.21.15.43:0/4268747208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc037070 con 0x7f3604160f40 2024-04-26T20:32:08.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.813+0000 7f3602000640 1 -- 172.21.15.43:0/4268747208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35fc032440 con 0x7f3604160f40 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.813+0000 7f3602000640 1 -- 172.21.15.43:0/4268747208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc032810 con 0x7f3604160f40 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.813+0000 7f3608c00640 1 -- 172.21.15.43:0/4268747208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604160f40 msgr2=0x7f360415c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.813+0000 7f3608c00640 1 --2- 172.21.15.43:0/4268747208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604160f40 0x7f360415c610 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0097a0 tx=0x7f35fc02a230 comp rx=0 tx=0).stop 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.814+0000 7f3608c00640 1 -- 172.21.15.43:0/4268747208 shutdown_connections 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.814+0000 7f3608c00640 1 --2- 172.21.15.43:0/4268747208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604160f40 0x7f360415c610 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.814+0000 7f3608c00640 1 -- 172.21.15.43:0/4268747208 >> 172.21.15.43:0/4268747208 conn(0x7f36040d29c0 msgr2=0x7f36040d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.814+0000 7f3608c00640 1 -- 172.21.15.43:0/4268747208 shutdown_connections 2024-04-26T20:32:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.814+0000 7f3608c00640 1 -- 172.21.15.43:0/4268747208 wait complete. 2024-04-26T20:32:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.815+0000 7f3608c00640 1 Processor -- start 2024-04-26T20:32:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.816+0000 7f3608c00640 1 -- start start 2024-04-26T20:32:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.816+0000 7f3608c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3608c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36040d36e0 0x7f36040c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3608c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3604160f40 0x7f36041f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3608c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36040ce880 con 0x7f3604071870 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3608c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f36040ce080 con 0x7f3604160f40 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3608c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f360415cb50 con 0x7f36040d36e0 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3603400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3603400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51774/0 (socket says 172.21.15.43:51774) 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3603400640 1 -- 172.21.15.43:0/3012328270 learned_addr learned my addr 172.21.15.43:0/3012328270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:08.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3603e00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3604160f40 0x7f36041f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.817+0000 7f3602a00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36040d36e0 0x7f36040c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 -- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36040d36e0 msgr2=0x7f36040c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36040d36e0 0x7f36040c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 -- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3604160f40 msgr2=0x7f36041f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3604160f40 0x7f36041f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 -- 172.21.15.43:0/3012328270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36040c9820 con 0x7f3604071870 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.818+0000 7f3603400640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c790 tx=0x7f35f400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.819+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4016020 con 0x7f3604071870 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.819+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f4006d50 con 0x7f3604071870 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.819+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36040c6590 con 0x7f3604071870 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.819+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4003bb0 con 0x7f3604071870 2024-04-26T20:32:08.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.819+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36040c7390 con 0x7f3604071870 2024-04-26T20:32:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.820+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3604069a60 con 0x7f3604071870 2024-04-26T20:32:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.821+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35f400a850 con 0x7f3604071870 2024-04-26T20:32:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.822+0000 7f35fb400640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 0x7f35e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:08.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.822+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f35f408b990 con 0x7f3604071870 2024-04-26T20:32:08.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.822+0000 7f3602a00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 0x7f35e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:08.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.823+0000 7f3602a00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 0x7f35e006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3604154d70 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:08.827+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35f4020020 con 0x7f3604071870 2024-04-26T20:32:09.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.073+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f36040cf1b0 con 0x7f3604071870 2024-04-26T20:32:09.075 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-26T20:32:09.074+0000 7f2bfa400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.077+0000 7f35fb400640 1 -- 172.21.15.43:0/3012328270 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f35f4059c00 con 0x7f3604071870 2024-04-26T20:32:09.079 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 msgr2=0x7f35e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 0x7f35e006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3604154d70 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 msgr2=0x7f36041541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c790 tx=0x7f35f400cc50 comp rx=0 tx=0).stop 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 shutdown_connections 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f35e006c8f0 0x7f35e006edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f36040d36e0 0x7f36040c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3604160f40 0x7f36041f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.080+0000 7f3608c00640 1 --2- 172.21.15.43:0/3012328270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3604071870 0x7f36041541b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.081+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 >> 172.21.15.43:0/3012328270 conn(0x7f36040d29c0 msgr2=0x7f36040b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.082+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 shutdown_connections 2024-04-26T20:32:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.082+0000 7f3608c00640 1 -- 172.21.15.43:0/3012328270 wait complete. 2024-04-26T20:32:09.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:32:09.165 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-26T20:32:09.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.293+0000 7f0309200640 1 Processor -- start 2024-04-26T20:32:09.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.294+0000 7f0309200640 1 -- start start 2024-04-26T20:32:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0309200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f030415f180 0x7f03040d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0309200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0309200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03040d2d70 con 0x7f03040d3660 2024-04-26T20:32:09.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0309200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f030415f560 con 0x7f030415f180 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0309200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f03040b2750 con 0x7f03040b3cb0 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0303400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f030415f180 0x7f03040d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0303400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f030415f180 0x7f03040d56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51800/0 (socket says 172.21.15.43:51800) 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.295+0000 7f0303400640 1 -- 172.21.15.43:0/2097954661 learned_addr learned my addr 172.21.15.43:0/2097954661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.296+0000 7f0303400640 1 -- 172.21.15.43:0/2097954661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 msgr2=0x7f03040d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.296+0000 7f0303400640 1 --2- 172.21.15.43:0/2097954661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.296+0000 7f0303400640 1 -- 172.21.15.43:0/2097954661 >> v1:172.21.15.164:6789/0 conn(0x7f03040d3660 legacy=0x7f03040d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.296+0000 7f0303400640 1 -- 172.21.15.43:0/2097954661 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03040d06e0 con 0x7f030415f180 2024-04-26T20:32:09.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.297+0000 7f0303400640 1 --2- 172.21.15.43:0/2097954661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030415f180 0x7f03040d56f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009870 tx=0x7f02f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9da9a0f160a07c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.297+0000 7f0302000640 1 -- 172.21.15.43:0/2097954661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4037070 con 0x7f030415f180 2024-04-26T20:32:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.297+0000 7f0302000640 1 -- 172.21.15.43:0/2097954661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f4032440 con 0x7f030415f180 2024-04-26T20:32:09.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0302000640 1 -- 172.21.15.43:0/2097954661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4032810 con 0x7f030415f180 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 -- 172.21.15.43:0/2097954661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030415f180 msgr2=0x7f03040d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 --2- 172.21.15.43:0/2097954661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030415f180 0x7f03040d56f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009870 tx=0x7f02f402a210 comp rx=0 tx=0).stop 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 -- 172.21.15.43:0/2097954661 shutdown_connections 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 --2- 172.21.15.43:0/2097954661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 --2- 172.21.15.43:0/2097954661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f030415f180 0x7f03040d56f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 -- 172.21.15.43:0/2097954661 >> 172.21.15.43:0/2097954661 conn(0x7f03040d4bd0 msgr2=0x7f03040d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 -- 172.21.15.43:0/2097954661 shutdown_connections 2024-04-26T20:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.298+0000 7f0309200640 1 -- 172.21.15.43:0/2097954661 wait complete. 2024-04-26T20:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.300+0000 7f0309200640 1 Processor -- start 2024-04-26T20:32:09.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.300+0000 7f0309200640 1 -- start start 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 0x7f03040ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f03040d2a40 con 0x7f03040d3660 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f03040d0ca0 con 0x7f030415f180 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.301+0000 7f0309200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f03040d11c0 con 0x7f03040b3cb0 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51808/0 (socket says 172.21.15.43:51808) 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 -- 172.21.15.43:0/518395856 learned_addr learned my addr 172.21.15.43:0/518395856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0303400640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0303e00640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 0x7f03040ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:09.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 -- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 msgr2=0x7f03040b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 -- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 msgr2=0x7f03040ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 0x7f03040ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 -- 172.21.15.43:0/518395856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03041f4c90 con 0x7f03040d3660 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0303e00640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 0x7f03040ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.302+0000 7f0302a00640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00b9c0 tx=0x7f02ec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.303+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec019070 con 0x7f03040d3660 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.303+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ec00cbc0 con 0x7f03040d3660 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.303+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec014440 con 0x7f03040d3660 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.303+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03040b6a60 con 0x7f03040d3660 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.303+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03040b5760 con 0x7f03040d3660 2024-04-26T20:32:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.304+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03040506c0 con 0x7f03040d3660 2024-04-26T20:32:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.306+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02ec01c070 con 0x7f03040d3660 2024-04-26T20:32:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.306+0000 7f02fb400640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 0x7f02e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.307+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f02ec08bab0 con 0x7f03040d3660 2024-04-26T20:32:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.307+0000 7f0303400640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 0x7f02e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:09.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.308+0000 7f0303400640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 0x7f02e006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02f4034000 tx=0x7f02f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:09.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.312+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02ec059cd0 con 0x7f03040d3660 2024-04-26T20:32:09.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:09.562+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f030415e580 con 0x7f03040d3660 2024-04-26T20:32:10.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.080+0000 7f02fb400640 1 -- 172.21.15.43:0/518395856 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f02ec0595d0 con 0x7f03040d3660 2024-04-26T20:32:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 msgr2=0x7f02e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 0x7f02e006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02f4034000 tx=0x7f02f402e000 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 msgr2=0x7f03040b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00b9c0 tx=0x7f02ec00be80 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 shutdown_connections 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f02e006c8f0 0x7f02e006edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f030415f180 0x7f03040ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03040d3660 0x7f03040b7a30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 --2- 172.21.15.43:0/518395856 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f03040b3cb0 0x7f03040b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.085+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 >> 172.21.15.43:0/518395856 conn(0x7f03040d4bd0 msgr2=0x7f03040cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:10.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.086+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 shutdown_connections 2024-04-26T20:32:10.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.086+0000 7f0309200640 1 -- 172.21.15.43:0/518395856 wait complete. 2024-04-26T20:32:10.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:32:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.319+0000 7fd5bc800640 1 Processor -- start 2024-04-26T20:32:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.319+0000 7fd5bc800640 1 -- start start 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.321+0000 7fd5bc800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd5b406f830 0x7fd5b406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.321+0000 7fd5bc800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.321+0000 7fd5bc800640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b4070150 con 0x7fd5b40b46d0 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.321+0000 7fd5bc800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b40b3c10 con 0x7fd5b406f830 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.321+0000 7fd5bc800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b415d180 con 0x7fd5b415c450 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5ba800640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd5b40b46d0 0x7fd5b40b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51946/0 (socket says 172.21.15.43:51946) 2024-04-26T20:32:10.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5ba800640 1 -- 172.21.15.43:0/949753733 learned_addr learned my addr 172.21.15.43:0/949753733 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5b9400640 1 --2- 172.21.15.43:0/949753733 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5b9e00640 1 --2- 172.21.15.43:0/949753733 >> v2:172.21.15.164:3300/0 conn(0x7fd5b406f830 0x7fd5b406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435250964 0 0) 0x7fd5b4070150 con 0x7fd5b40b46d0 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.322+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd59c003520 con 0x7fd5b40b46d0 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.323+0000 7fd5b9400640 1 -- 172.21.15.43:0/949753733 >> v2:172.21.15.164:3300/0 conn(0x7fd5b406f830 msgr2=0x7fd5b406fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.323+0000 7fd5b9400640 1 --2- 172.21.15.43:0/949753733 >> v2:172.21.15.164:3300/0 conn(0x7fd5b406f830 0x7fd5b406fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.323+0000 7fd5b9400640 1 -- 172.21.15.43:0/949753733 >> v1:172.21.15.43:6789/0 conn(0x7fd5b40b46d0 legacy=0x7fd5b40b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.323+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1549722425 0 0) 0x7fd59c003520 con 0x7fd5b40b46d0 2024-04-26T20:32:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.323+0000 7fd5b9400640 1 -- 172.21.15.43:0/949753733 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b40b3ed0 con 0x7fd5b415c450 2024-04-26T20:32:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.324+0000 7fd5b9400640 1 --2- 172.21.15.43:0/949753733 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b415c8d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5aa12288a76c0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.325+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4037070 con 0x7fd5b415c450 2024-04-26T20:32:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.325+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a402ab10 con 0x7fd5b415c450 2024-04-26T20:32:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.325+0000 7fd5b8a00640 1 -- 172.21.15.43:0/949753733 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00019f0 con 0x7fd5b415c450 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.325+0000 7fd5bc800640 1 -- 172.21.15.43:0/949753733 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 msgr2=0x7fd5b415c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.325+0000 7fd5bc800640 1 --2- 172.21.15.43:0/949753733 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b415c8d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402a080 comp rx=0 tx=0).stop 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 -- 172.21.15.43:0/949753733 shutdown_connections 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 --2- 172.21.15.43:0/949753733 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b415c8d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 --2- 172.21.15.43:0/949753733 >> v2:172.21.15.164:3300/0 conn(0x7fd5b406f830 0x7fd5b406fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 -- 172.21.15.43:0/949753733 >> 172.21.15.43:0/949753733 conn(0x7fd5b40b2790 msgr2=0x7fd5b40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:10.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 -- 172.21.15.43:0/949753733 shutdown_connections 2024-04-26T20:32:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.326+0000 7fd5bc800640 1 -- 172.21.15.43:0/949753733 wait complete. 2024-04-26T20:32:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.327+0000 7fd5bc800640 1 Processor -- start 2024-04-26T20:32:10.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.328+0000 7fd5bc800640 1 -- start start 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.328+0000 7fd5bc800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 0x7fd5b4138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.328+0000 7fd5bc800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5bc800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5bc800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40738a0 con 0x7fd5b40b46d0 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5bc800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b4073720 con 0x7fd5b406f830 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5bc800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b40b5270 con 0x7fd5b415c450 2024-04-26T20:32:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51820/0 (socket says 172.21.15.43:51820) 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 -- 172.21.15.43:0/1875499137 learned_addr learned my addr 172.21.15.43:0/1875499137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9e00640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 0x7fd5b4138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 -- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 msgr2=0x7fd5b414c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 -- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 msgr2=0x7fd5b4138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 0x7fd5b4138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9400640 1 -- 172.21.15.43:0/1875499137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b4138a40 con 0x7fd5b40b46d0 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.329+0000 7fd5b9e00640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 0x7fd5b4138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5b9400640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4035040 tx=0x7fd5a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4041020 con 0x7fd5b40b46d0 2024-04-26T20:32:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a403d640 con 0x7fd5b40b46d0 2024-04-26T20:32:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4037040 con 0x7fd5b40b46d0 2024-04-26T20:32:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b4158490 con 0x7fd5b40b46d0 2024-04-26T20:32:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.330+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b4158a50 con 0x7fd5b40b46d0 2024-04-26T20:32:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.332+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b4069350 con 0x7fd5b40b46d0 2024-04-26T20:32:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.333+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5a4044020 con 0x7fd5b40b46d0 2024-04-26T20:32:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.334+0000 7fd5ae000640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 0x7fd59806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.334+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fd5a40ac1d0 con 0x7fd5b40b46d0 2024-04-26T20:32:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.334+0000 7fd5b9e00640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 0x7fd59806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:10.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.335+0000 7fd5b9e00640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 0x7fd59806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd5b414cfc0 tx=0x7fd5a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:10.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.338+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5a404b030 con 0x7fd5b40b46d0 2024-04-26T20:32:10.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:10.543+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd5b40b5690 con 0x7fd5b40b46d0 2024-04-26T20:32:11.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.084+0000 7fd5ae000640 1 -- 172.21.15.43:0/1875499137 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7fd5b4138a40 con 0x7fd5b40b46d0 2024-04-26T20:32:11.086 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.087+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 msgr2=0x7fd59806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.087+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 0x7fd59806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd5b414cfc0 tx=0x7fd5a8008040 comp rx=0 tx=0).stop 2024-04-26T20:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 msgr2=0x7fd5b414c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4035040 tx=0x7fd5a40046c0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 shutdown_connections 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fd59806c920 0x7fd59806ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5b415c450 0x7fd5b414c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b46d0 0x7fd5b414c370 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 --2- 172.21.15.43:0/1875499137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5b406f830 0x7fd5b4138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 >> 172.21.15.43:0/1875499137 conn(0x7fd5b40b2790 msgr2=0x7fd5b40cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 shutdown_connections 2024-04-26T20:32:11.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.088+0000 7fd5bc800640 1 -- 172.21.15.43:0/1875499137 wait complete. 2024-04-26T20:32:11.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:32:11.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.333+0000 7fa035600640 1 Processor -- start 2024-04-26T20:32:11.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.333+0000 7fa035600640 1 -- start start 2024-04-26T20:32:11.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.335+0000 7fa035600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa030146800 0x7fa0300cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.335+0000 7fa035600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 0x7fa030072290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.335+0000 7fa035600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03006df30 con 0x7fa03006fec0 2024-04-26T20:32:11.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.335+0000 7fa035600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa030146be0 con 0x7fa030146800 2024-04-26T20:32:11.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.335+0000 7fa035600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa03006e860 con 0x7fa030071e60 2024-04-26T20:32:11.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.336+0000 7fa02f400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 0x7fa030072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.336+0000 7fa02e000640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa03006fec0 0x7fa03006ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.43:53778/0 (socket says 172.21.15.43:53778) 2024-04-26T20:32:11.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.336+0000 7fa02e000640 1 -- 172.21.15.43:0/3100861308 learned_addr learned my addr 172.21.15.43:0/3100861308 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:11.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.336+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3100861308 >> v2:172.21.15.164:3300/0 conn(0x7fa030146800 0x7fa0300cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.337+0000 7fa02ea00640 1 -- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 msgr2=0x7fa030072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:11.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.337+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 0x7fa030072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.337+0000 7fa02ea00640 1 -- 172.21.15.43:0/3100861308 >> v1:172.21.15.164:6789/0 conn(0x7fa03006fec0 legacy=0x7fa03006ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.337+0000 7fa02ea00640 1 -- 172.21.15.43:0/3100861308 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03006e210 con 0x7fa030146800 2024-04-26T20:32:11.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.337+0000 7fa02f400640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 0x7fa030072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:32:11.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.339+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa0300cfcd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0200097a0 tx=0x7fa02002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4749f1ac720c7116 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:11.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.339+0000 7fa02d600640 1 -- 172.21.15.43:0/3100861308 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020037070 con 0x7fa030146800 2024-04-26T20:32:11.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.339+0000 7fa02d600640 1 -- 172.21.15.43:0/3100861308 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa020032440 con 0x7fa030146800 2024-04-26T20:32:11.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.339+0000 7fa02d600640 1 -- 172.21.15.43:0/3100861308 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020032810 con 0x7fa030146800 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 -- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 msgr2=0x7fa0300cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa0300cfcd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0200097a0 tx=0x7fa02002a180 comp rx=0 tx=0).stop 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 -- 172.21.15.43:0/3100861308 shutdown_connections 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa030071e60 0x7fa030072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 --2- 172.21.15.43:0/3100861308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa0300cfcd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 -- 172.21.15.43:0/3100861308 >> 172.21.15.43:0/3100861308 conn(0x7fa0300b2790 msgr2=0x7fa0300b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:11.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.340+0000 7fa035600640 1 -- 172.21.15.43:0/3100861308 shutdown_connections 2024-04-26T20:32:11.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.341+0000 7fa035600640 1 -- 172.21.15.43:0/3100861308 wait complete. 2024-04-26T20:32:11.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.342+0000 7fa035600640 1 Processor -- start 2024-04-26T20:32:11.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.342+0000 7fa035600640 1 -- start start 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 0x7fa0301396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa03013af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa0300d0520 con 0x7fa030071e60 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0300d0210 con 0x7fa030146800 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.343+0000 7fa035600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa03006f400 con 0x7fa03006fec0 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51830/0 (socket says 172.21.15.43:51830) 2024-04-26T20:32:11.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 -- 172.21.15.43:0/3631461167 learned_addr learned my addr 172.21.15.43:0/3631461167 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 0x7fa0301396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02f400640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa03013af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 -- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 msgr2=0x7fa0301396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 0x7fa0301396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 -- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 msgr2=0x7fa03013af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa03013af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 -- 172.21.15.43:0/3631461167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03013b960 con 0x7fa030071e60 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 0x7fa0301396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.344+0000 7fa02e000640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cb10 tx=0x7fa01800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.345+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018017070 con 0x7fa030071e60 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.345+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01800ac70 con 0x7fa030071e60 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.345+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ae60 con 0x7fa030071e60 2024-04-26T20:32:11.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.345+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03013bc40 con 0x7fa030071e60 2024-04-26T20:32:11.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.345+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03013e320 con 0x7fa030071e60 2024-04-26T20:32:11.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.346+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa030069350 con 0x7fa030071e60 2024-04-26T20:32:11.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.347+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa01801a070 con 0x7fa030071e60 2024-04-26T20:32:11.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.348+0000 7fa026a00640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 0x7fa00c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:11.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.348+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5778+0+0 (secure 0 0 0) 0x7fa01808ba30 con 0x7fa030071e60 2024-04-26T20:32:11.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.348+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 0x7fa00c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:11.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.349+0000 7fa02ea00640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 0x7fa00c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa020034000 tx=0x7fa02002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:11.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.352+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa018059ce0 con 0x7fa030071e60 2024-04-26T20:32:11.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:11.555+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa03013be10 con 0x7fa030071e60 2024-04-26T20:32:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.096+0000 7fa026a00640 1 -- 172.21.15.43:0/3631461167 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7fa0180595e0 con 0x7fa030071e60 2024-04-26T20:32:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 msgr2=0x7fa00c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 0x7fa00c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa020034000 tx=0x7fa02002e000 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 msgr2=0x7fa03013dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cb10 tx=0x7fa01800a060 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 shutdown_connections 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa00c06c8f0 0x7fa00c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa030146800 0x7fa03013af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa030071e60 0x7fa03013dde0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 --2- 172.21.15.43:0/3631461167 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa03006fec0 0x7fa0301396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.100+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 >> 172.21.15.43:0/3631461167 conn(0x7fa0300b2790 msgr2=0x7fa030140180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.101+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 shutdown_connections 2024-04-26T20:32:12.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.102+0000 7fa035600640 1 -- 172.21.15.43:0/3631461167 wait complete. 2024-04-26T20:32:12.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:32:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.337+0000 7fc9f8c00640 1 Processor -- start 2024-04-26T20:32:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.338+0000 7fc9f8c00640 1 -- start start 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f8c00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9f415f2a0 0x7fc9f40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f8c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f40d2d60 con 0x7fc9f40d3210 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f8c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f40d2a80 con 0x7fc9f40d35f0 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f8c00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f40d0a20 con 0x7fc9f415f2a0 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f3400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc9f40d3210 0x7fc9f415eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51958/0 (socket says 172.21.15.43:51958) 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.339+0000 7fc9f3400640 1 -- 172.21.15.43:0/2315659483 learned_addr learned my addr 172.21.15.43:0/2315659483 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f3e00640 1 --2- 172.21.15.43:0/2315659483 >> v2:172.21.15.43:3300/0 conn(0x7fc9f415f2a0 0x7fc9f40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f2a00640 1 -- 172.21.15.43:0/2315659483 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365914504 0 0) 0x7fc9f40d2d60 con 0x7fc9f40d3210 2024-04-26T20:32:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f2a00640 1 -- 172.21.15.43:0/2315659483 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9d8003520 con 0x7fc9f40d3210 2024-04-26T20:32:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f3e00640 1 -- 172.21.15.43:0/2315659483 >> v1:172.21.15.164:6789/0 conn(0x7fc9f40d35f0 legacy=0x7fc9f406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f3e00640 1 -- 172.21.15.43:0/2315659483 >> v1:172.21.15.43:6789/0 conn(0x7fc9f40d3210 legacy=0x7fc9f415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.340+0000 7fc9f3e00640 1 -- 172.21.15.43:0/2315659483 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f40d03d0 con 0x7fc9f415f2a0 2024-04-26T20:32:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.342+0000 7fc9f3e00640 1 --2- 172.21.15.43:0/2315659483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b41f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009d90 tx=0x7fc9ec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa377662b5a0b46a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.342+0000 7fc9f2a00640 1 -- 172.21.15.43:0/2315659483 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec007740 con 0x7fc9f415f2a0 2024-04-26T20:32:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.342+0000 7fc9f2a00640 1 -- 172.21.15.43:0/2315659483 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0078e0 con 0x7fc9f415f2a0 2024-04-26T20:32:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.342+0000 7fc9f2a00640 1 -- 172.21.15.43:0/2315659483 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec007c90 con 0x7fc9f415f2a0 2024-04-26T20:32:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 -- 172.21.15.43:0/2315659483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f415f2a0 msgr2=0x7fc9f40b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/2315659483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b41f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009d90 tx=0x7fc9ec02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 -- 172.21.15.43:0/2315659483 shutdown_connections 2024-04-26T20:32:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/2315659483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b41f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 -- 172.21.15.43:0/2315659483 >> 172.21.15.43:0/2315659483 conn(0x7fc9f40d4d10 msgr2=0x7fc9f40d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 -- 172.21.15.43:0/2315659483 shutdown_connections 2024-04-26T20:32:12.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.343+0000 7fc9f8c00640 1 -- 172.21.15.43:0/2315659483 wait complete. 2024-04-26T20:32:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.345+0000 7fc9f8c00640 1 Processor -- start 2024-04-26T20:32:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.345+0000 7fc9f8c00640 1 -- start start 2024-04-26T20:32:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 0x7fc9f40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f40d59b0 con 0x7fc9f40d35f0 2024-04-26T20:32:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f40d5830 con 0x7fc9f415f2a0 2024-04-26T20:32:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f8c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f40d5b30 con 0x7fc9f40d3210 2024-04-26T20:32:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51846/0 (socket says 172.21.15.43:51846) 2024-04-26T20:32:12.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9ebe00640 1 -- 172.21.15.43:0/1029467380 learned_addr learned my addr 172.21.15.43:0/1029467380 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f3e00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9f3400640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 0x7fc9f40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.346+0000 7fc9ebe00640 1 -- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 msgr2=0x7fc9f40b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9ebe00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 0x7fc9f40b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9ebe00640 1 -- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 msgr2=0x7fc9f40b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9ebe00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9ebe00640 1 -- 172.21.15.43:0/1029467380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f40b7f80 con 0x7fc9f40d35f0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9f3e00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9f3400640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 0x7fc9f40b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9ebe00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c450 tx=0x7fc9dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0052d0 con 0x7fc9f40d35f0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc005470 con 0x7fc9f40d35f0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc012870 con 0x7fc9f40d35f0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f40b8260 con 0x7fc9f40d35f0 2024-04-26T20:32:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.347+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f40b8820 con 0x7fc9f40d35f0 2024-04-26T20:32:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.349+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f40506c0 con 0x7fc9f40d35f0 2024-04-26T20:32:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.349+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9dc012a10 con 0x7fc9f40d35f0 2024-04-26T20:32:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.351+0000 7fc9eb400640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 0x7fc9d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.351+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc9dc08b680 con 0x7fc9f40d35f0 2024-04-26T20:32:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.351+0000 7fc9f3400640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 0x7fc9d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.352+0000 7fc9f3400640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 0x7fc9d006edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40011e0 tx=0x7fc9e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.356+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc9dc059b80 con 0x7fc9f40d35f0 2024-04-26T20:32:12.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.559+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc9f40d5290 con 0x7fc9f40d35f0 2024-04-26T20:32:12.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.564+0000 7fc9eb400640 1 -- 172.21.15.43:0/1029467380 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fc9dc059480 con 0x7fc9f40d35f0 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 msgr2=0x7fc9d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 0x7fc9d006edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40011e0 tx=0x7fc9e4009040 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 msgr2=0x7fc9f40ba1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c450 tx=0x7fc9dc00c910 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 shutdown_connections 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc9d006c8f0 0x7fc9d006edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc9f415f2a0 0x7fc9f40b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9f40d35f0 0x7fc9f40ba1b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 --2- 172.21.15.43:0/1029467380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc9f40d3210 0x7fc9f40b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 >> 172.21.15.43:0/1029467380 conn(0x7fc9f40d4d10 msgr2=0x7fc9f40d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 shutdown_connections 2024-04-26T20:32:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.567+0000 7fc9f8c00640 1 -- 172.21.15.43:0/1029467380 wait complete. 2024-04-26T20:32:12.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-26T20:32:12.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.799+0000 7fc602c00640 1 Processor -- start 2024-04-26T20:32:12.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.800+0000 7fc602c00640 1 -- start start 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.801+0000 7fc602c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5fc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.801+0000 7fc602c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc0cf060 con 0x7fc5fc06f730 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.801+0000 7fc602c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5fc0cf340 con 0x7fc5fc0b3490 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.801+0000 7fc602c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0d0100 con 0x7fc5fc145590 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.802+0000 7fc600c00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc5fc06f730 0x7fc5fc071eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:51968/0 (socket says 172.21.15.43:51968) 2024-04-26T20:32:12.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.802+0000 7fc600c00640 1 -- 172.21.15.43:0/3265362547 learned_addr learned my addr 172.21.15.43:0/3265362547 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:12.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.802+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 328685918 0 0) 0x7fc5fc0cf060 con 0x7fc5fc06f730 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.802+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5e0003520 con 0x7fc5fc06f730 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.802+0000 7fc5fbe00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5fc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711843471 0 0) 0x7fc5e0003520 con 0x7fc5fc06f730 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 msgr2=0x7fc5fc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5fc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 >> v1:172.21.15.164:6789/0 conn(0x7fc5fc0b3490 legacy=0x7fc5fc06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc0cf620 con 0x7fc5fc06f730 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fc5f40019f0 con 0x7fc5fc06f730 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 >> v1:172.21.15.43:6789/0 conn(0x7fc5fc06f730 legacy=0x7fc5fc071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e00039e0 0x7fc5e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0004800 0x7fc5e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc0d0100 con 0x7fc5e0004800 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e000baf0 con 0x7fc5e00039e0 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e000df00 con 0x7fc5fc145590 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.803+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0004800 0x7fc5e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5fbe00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e00039e0 0x7fc5e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc600c00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 msgr2=0x7fc5e0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc145590 0x7fc5e0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e00039e0 msgr2=0x7fc5e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e00039e0 0x7fc5e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.804+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3265362547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e0003520 con 0x7fc5e0004800 2024-04-26T20:32:12.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.805+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0004800 0x7fc5e0004c80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001720 tx=0x7fc5e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5d5be639054bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.805+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4001370 con 0x7fc5e0004800 2024-04-26T20:32:12.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.805+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e40052d0 con 0x7fc5e0004800 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.805+0000 7fc5fb400640 1 -- 172.21.15.43:0/3265362547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e402abc0 con 0x7fc5e0004800 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 -- 172.21.15.43:0/3265362547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0004800 msgr2=0x7fc5e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 --2- 172.21.15.43:0/3265362547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5e0004800 0x7fc5e0004c80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001720 tx=0x7fc5e0005830 comp rx=0 tx=0).stop 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc600c00640 1 -- 172.21.15.43:0/3265362547 reap_dead start 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 -- 172.21.15.43:0/3265362547 shutdown_connections 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 -- 172.21.15.43:0/3265362547 >> 172.21.15.43:0/3265362547 conn(0x7fc5fc0b2900 msgr2=0x7fc5fc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 -- 172.21.15.43:0/3265362547 shutdown_connections 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.806+0000 7fc602c00640 1 -- 172.21.15.43:0/3265362547 wait complete. 2024-04-26T20:32:12.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.808+0000 7fc602c00640 1 Processor -- start 2024-04-26T20:32:12.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.808+0000 7fc602c00640 1 -- start start 2024-04-26T20:32:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e0004800 0x7fc5fc13a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc0af910 0x7fc5fc0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc145970 con 0x7fc5fc159ed0 2024-04-26T20:32:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc0cfbe0 con 0x7fc5e0004800 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc602c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0ce780 con 0x7fc5fc0af910 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.809+0000 7fc5f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51874/0 (socket says 172.21.15.43:51874) 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3932160288 learned_addr learned my addr 172.21.15.43:0/3932160288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5fbe00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e0004800 0x7fc5fc13a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc0af910 msgr2=0x7fc5fc0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc0af910 0x7fc5fc0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e0004800 msgr2=0x7fc5fc13a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e0004800 0x7fc5fc13a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 -- 172.21.15.43:0/3932160288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc13abd0 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.810+0000 7fc5f3e00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4008fd0 tx=0x7fc5e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.811+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4042070 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.811+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e4005480 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.811+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4037040 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.811+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc13aeb0 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.811+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc13b470 con 0x7fc5fc159ed0 2024-04-26T20:32:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.812+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc0b02d0 con 0x7fc5fc159ed0 2024-04-26T20:32:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.813+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5e4045070 con 0x7fc5fc159ed0 2024-04-26T20:32:12.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.814+0000 7fc5f8c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 0x7fc5d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:12.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.814+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc5e40ab260 con 0x7fc5fc159ed0 2024-04-26T20:32:12.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.814+0000 7fc5fbe00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 0x7fc5d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:12.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.815+0000 7fc5fbe00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 0x7fc5d806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006fd0 tx=0x7fc5ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:12.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:12.819+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5e4079760 con 0x7fc5fc159ed0 2024-04-26T20:32:13.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:13.102+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fc5fc076260 con 0x7fc5fc159ed0 2024-04-26T20:32:14.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.102+0000 7fc5f8c00640 1 -- 172.21.15.43:0/3932160288 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v268) ==== 120+0+0 (secure 0 0 0) 0x7fc5fc076260 con 0x7fc5fc159ed0 2024-04-26T20:32:14.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.106+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 msgr2=0x7fc5d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:14.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.106+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 0x7fc5d806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006fd0 tx=0x7fc5ec008040 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.106+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 msgr2=0x7fc5fc142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.106+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4008fd0 tx=0x7fc5e4001120 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 shutdown_connections 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fc5d806c8f0 0x7fc5d806edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc5fc0af910 0x7fc5fc0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5fc159ed0 0x7fc5fc142cd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 --2- 172.21.15.43:0/3932160288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5e0004800 0x7fc5fc13a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 >> 172.21.15.43:0/3932160288 conn(0x7fc5fc0b2900 msgr2=0x7fc5fc143990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 shutdown_connections 2024-04-26T20:32:14.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:14.107+0000 7fc602c00640 1 -- 172.21.15.43:0/3932160288 wait complete. 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner: 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner:Ran 1 test in 311.004s 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner: 2024-04-26T20:32:14.139 INFO:tasks.cephfs_test_runner:OK 2024-04-26T20:32:14.140 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T20:32:14.150 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T20:32:14.158 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T20:32:14.159 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:32:14.159 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:14.206 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:32:14.206 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-26T20:32:14.207 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:14.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:32:14.305 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:32:14.305 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:14.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:32:14.363 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-26T20:32:14.363 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:32:14.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:32:14.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.418 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-04-26T20:32:14.472 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:32:14.472 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:32:14.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:32:14.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.558 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:32:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-04-26T20:32:14.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.767 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-26T20:32:14.827 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:32:14.828 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:32:14.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:32:14.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:14.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:32:15.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:32:15.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-26T20:32:15.127 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T20:32:15.137 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:32:15.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:32:15.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.319+0000 7fa32b200640 1 Processor -- start 2024-04-26T20:32:15.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.319+0000 7fa32b200640 1 -- start start 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0cd180 0x7fa32c072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa32c15dc50 0x7fa32c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa32c0ce2b0 0x7fa32c1607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa32c0cd560 con 0x7fa32c15dc50 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa32c072d00 con 0x7fa32c0ce2b0 2024-04-26T20:32:15.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa32b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c072fe0 con 0x7fa32c0cd180 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa323e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa32c15dc50 0x7fa32c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa323e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa32c15dc50 0x7fa32c0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51890/0 (socket says 172.21.15.43:51890) 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa323e00640 1 -- 172.21.15.43:0/2196473877 learned_addr learned my addr 172.21.15.43:0/2196473877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.320+0000 7fa329e00640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0cd180 0x7fa32c072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa32a800640 1 --2- 172.21.15.43:0/2196473877 >> v2:172.21.15.164:3300/0 conn(0x7fa32c0ce2b0 0x7fa32c1607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa323e00640 1 -- 172.21.15.43:0/2196473877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0cd180 msgr2=0x7fa32c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa323e00640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0cd180 0x7fa32c072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa323e00640 1 -- 172.21.15.43:0/2196473877 >> v2:172.21.15.164:3300/0 conn(0x7fa32c0ce2b0 msgr2=0x7fa32c1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa323e00640 1 --2- 172.21.15.43:0/2196473877 >> v2:172.21.15.164:3300/0 conn(0x7fa32c0ce2b0 0x7fa32c1607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.321+0000 7fa323e00640 1 -- 172.21.15.43:0/2196473877 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c0732c0 con 0x7fa32c15dc50 2024-04-26T20:32:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.322+0000 7fa323e00640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c15dc50 0x7fa32c0cdd70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa32c072080 tx=0x7fa31402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5915cef36935cedd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.322+0000 7fa329400640 1 -- 172.21.15.43:0/2196473877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314037070 con 0x7fa32c15dc50 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.322+0000 7fa329400640 1 -- 172.21.15.43:0/2196473877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314032440 con 0x7fa32c15dc50 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.322+0000 7fa329400640 1 -- 172.21.15.43:0/2196473877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314032810 con 0x7fa32c15dc50 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 -- 172.21.15.43:0/2196473877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c15dc50 msgr2=0x7fa32c0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c15dc50 0x7fa32c0cdd70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa32c072080 tx=0x7fa31402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 -- 172.21.15.43:0/2196473877 shutdown_connections 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 --2- 172.21.15.43:0/2196473877 >> v2:172.21.15.164:3300/0 conn(0x7fa32c0ce2b0 0x7fa32c1607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c15dc50 0x7fa32c0cdd70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 --2- 172.21.15.43:0/2196473877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0cd180 0x7fa32c072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 -- 172.21.15.43:0/2196473877 >> 172.21.15.43:0/2196473877 conn(0x7fa32c068090 msgr2=0x7fa32c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 -- 172.21.15.43:0/2196473877 shutdown_connections 2024-04-26T20:32:15.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.323+0000 7fa32b200640 1 -- 172.21.15.43:0/2196473877 wait complete. 2024-04-26T20:32:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.325+0000 7fa32b200640 1 Processor -- start 2024-04-26T20:32:15.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.325+0000 7fa32b200640 1 -- start start 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0ce2b0 0x7fa32c1f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa32c15dc50 0x7fa32c1fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c15e0d0 con 0x7fa32c0cd180 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c073da0 con 0x7fa32c15dc50 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa32b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c160d10 con 0x7fa32c0ce2b0 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa329e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa329e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51892/0 (socket says 172.21.15.43:51892) 2024-04-26T20:32:15.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.326+0000 7fa329e00640 1 -- 172.21.15.43:0/551351000 learned_addr learned my addr 172.21.15.43:0/551351000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa323e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0ce2b0 0x7fa32c1f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa32a800640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa32c15dc50 0x7fa32c1fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 -- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0ce2b0 msgr2=0x7fa32c1f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0ce2b0 0x7fa32c1f9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 -- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa32c15dc50 msgr2=0x7fa32c1fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa32c15dc50 0x7fa32c1fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 -- 172.21.15.43:0/551351000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c1fa8d0 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa329e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00d800 tx=0x7fa31c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c018020 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c0096a0 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c009a70 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.327+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c1fabb0 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.328+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c1fb170 con 0x7fa32c0cd180 2024-04-26T20:32:15.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.329+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32c069350 con 0x7fa32c0cd180 2024-04-26T20:32:15.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.330+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa31c01b020 con 0x7fa32c0cd180 2024-04-26T20:32:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.330+0000 7fa322000640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 0x7fa30806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.331+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fa31c08bc80 con 0x7fa32c0cd180 2024-04-26T20:32:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.331+0000 7fa323e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 0x7fa30806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.332+0000 7fa323e00640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 0x7fa30806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa314034000 tx=0x7fa31402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.335+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa31c022030 con 0x7fa32c0cd180 2024-04-26T20:32:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.536+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa32c068960 con 0x7fa30806c8f0 2024-04-26T20:32:15.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.539+0000 7fa322000640 1 -- 172.21.15.43:0/551351000 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7fa32c068960 con 0x7fa30806c8f0 2024-04-26T20:32:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:32:15.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 msgr2=0x7fa30806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 0x7fa30806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa314034000 tx=0x7fa31402e000 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 msgr2=0x7fa32c13aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00d800 tx=0x7fa31c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 shutdown_connections 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fa30806c8f0 0x7fa30806edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa32c0ce2b0 0x7fa32c1f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa32c15dc50 0x7fa32c1fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 --2- 172.21.15.43:0/551351000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c0cd180 0x7fa32c13aff0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.541+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 >> 172.21.15.43:0/551351000 conn(0x7fa32c068090 msgr2=0x7fa32c15f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:15.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.542+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 shutdown_connections 2024-04-26T20:32:15.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.542+0000 7fa32b200640 1 -- 172.21.15.43:0/551351000 wait complete. 2024-04-26T20:32:15.561 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-04-26T20:32:14.990289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237936,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736784,"statfs":{"total":773094113280,"available":772850466816,"internally_reserved":0,"allocated":5251072,"data_stored":2955352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002215"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:32:14.127842+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:32:14.127842+0000","last_peered":"2024-04-26T20:32:14.127842+0000","last_clean":"2024-04-26T20:32:14.127842+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:32:14.127842+0000","last_undegraded":"2024-04-26T20:32:14.127842+0000","last_fullsized":"2024-04-26T20:32:14.127842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:32:15.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:32:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.772+0000 7f6f07200640 1 Processor -- start 2024-04-26T20:32:15.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.772+0000 7f6f07200640 1 -- start start 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f07200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6f000d1620 0x7f6f000d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f07200640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f000d1a00 0x7f6f000b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f07200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f000d0480 con 0x7f6f000b50b0 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f07200640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6f000d0ce0 con 0x7f6f000d1a00 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f07200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6f000d0720 con 0x7f6f000d1620 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.773+0000 7f6f05400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f000d1a00 0x7f6f000b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f000d1a00 0x7f6f000b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51906/0 (socket says 172.21.15.43:51906) 2024-04-26T20:32:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 -- 172.21.15.43:0/3659304030 learned_addr learned my addr 172.21.15.43:0/3659304030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 -- 172.21.15.43:0/3659304030 >> v2:172.21.15.164:3300/0 conn(0x7f6f000d1620 msgr2=0x7f6f000d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 --2- 172.21.15.43:0/3659304030 >> v2:172.21.15.164:3300/0 conn(0x7f6f000d1620 0x7f6f000d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 -- 172.21.15.43:0/3659304030 >> v1:172.21.15.164:6789/0 conn(0x7f6f000b50b0 legacy=0x7f6f000b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:15.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.774+0000 7f6f05400640 1 -- 172.21.15.43:0/3659304030 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000d0a00 con 0x7f6f000d1a00 2024-04-26T20:32:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.775+0000 7f6f05400640 1 --2- 172.21.15.43:0/3659304030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000d1a00 0x7f6f000b4b70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d2730 tx=0x7f6ef002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fa99a46b1053c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.776+0000 7f6f04a00640 1 -- 172.21.15.43:0/3659304030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0037070 con 0x7f6f000d1a00 2024-04-26T20:32:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.776+0000 7f6f04a00640 1 -- 172.21.15.43:0/3659304030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef0032440 con 0x7f6f000d1a00 2024-04-26T20:32:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.776+0000 7f6f04a00640 1 -- 172.21.15.43:0/3659304030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0032810 con 0x7f6f000d1a00 2024-04-26T20:32:15.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 -- 172.21.15.43:0/3659304030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000d1a00 msgr2=0x7f6f000b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 --2- 172.21.15.43:0/3659304030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000d1a00 0x7f6f000b4b70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d2730 tx=0x7f6ef002a160 comp rx=0 tx=0).stop 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 -- 172.21.15.43:0/3659304030 shutdown_connections 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 --2- 172.21.15.43:0/3659304030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000d1a00 0x7f6f000b4b70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 --2- 172.21.15.43:0/3659304030 >> v2:172.21.15.164:3300/0 conn(0x7f6f000d1620 0x7f6f000d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 -- 172.21.15.43:0/3659304030 >> 172.21.15.43:0/3659304030 conn(0x7f6f000d4c30 msgr2=0x7f6f000d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 -- 172.21.15.43:0/3659304030 shutdown_connections 2024-04-26T20:32:15.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.777+0000 7f6f07200640 1 -- 172.21.15.43:0/3659304030 wait complete. 2024-04-26T20:32:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.779+0000 7f6f07200640 1 Processor -- start 2024-04-26T20:32:15.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.779+0000 7f6f07200640 1 -- start start 2024-04-26T20:32:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f000d1620 0x7f6f0015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 0x7f6f0015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000b5c50 con 0x7f6f000b50b0 2024-04-26T20:32:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000d1e60 con 0x7f6f000d1a00 2024-04-26T20:32:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f07200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6f000b5dd0 con 0x7f6f000d1620 2024-04-26T20:32:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51916/0 (socket says 172.21.15.43:51916) 2024-04-26T20:32:15.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 -- 172.21.15.43:0/3286658598 learned_addr learned my addr 172.21.15.43:0/3286658598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05400640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f000d1620 0x7f6f0015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 -- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f000d1620 msgr2=0x7f6f0015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f000d1620 0x7f6f0015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f05e00640 1 -- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 msgr2=0x7f6f0015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.780+0000 7f6f06800640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 0x7f6f0015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f05e00640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 0x7f6f0015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:15.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f05e00640 1 -- 172.21.15.43:0/3286658598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000ce730 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f06800640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 0x7f6f0015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f05e00640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b4d0 tx=0x7f6ef400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4014070 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f000c9880 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4010870 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4010c60 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.781+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f000cc960 con 0x7f6f000b50b0 2024-04-26T20:32:15.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.783+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f00069350 con 0x7f6f000b50b0 2024-04-26T20:32:15.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.784+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ef4017070 con 0x7f6f000b50b0 2024-04-26T20:32:15.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.785+0000 7f6efa000640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 0x7f6ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:15.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.785+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f6ef408b8f0 con 0x7f6f000b50b0 2024-04-26T20:32:15.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.785+0000 7f6f05400640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 0x7f6ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:15.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.786+0000 7f6f05400640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 0x7f6ee406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0034000 tx=0x7f6ef002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:15.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.790+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ef4059d70 con 0x7f6f000b50b0 2024-04-26T20:32:15.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.994+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f000d3bc0 con 0x7f6ee406c8f0 2024-04-26T20:32:15.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.997+0000 7f6efa000640 1 -- 172.21.15.43:0/3286658598 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f6f000d3bc0 con 0x7f6ee406c8f0 2024-04-26T20:32:15.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:15.998 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:32:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 msgr2=0x7f6ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 0x7f6ee406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0034000 tx=0x7f6ef002e000 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 msgr2=0x7f6f000cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400b4d0 tx=0x7f6ef400b990 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 shutdown_connections 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6ee406c8f0 0x7f6ee406edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6f000d1620 0x7f6f0015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6f000d1a00 0x7f6f0015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 --2- 172.21.15.43:0/3286658598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f000b50b0 0x7f6f000cd160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:15.999+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 >> 172.21.15.43:0/3286658598 conn(0x7f6f000d4c30 msgr2=0x7f6f001e8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.000+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 shutdown_connections 2024-04-26T20:32:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.000+0000 7f6f07200640 1 -- 172.21.15.43:0/3286658598 wait complete. 2024-04-26T20:32:16.021 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-04-26T20:32:14.990289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237936,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736784,"statfs":{"total":773094113280,"available":772850466816,"internally_reserved":0,"allocated":5251072,"data_stored":2955352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002215"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:32:14.127842+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:32:14.127842+0000","last_peered":"2024-04-26T20:32:14.127842+0000","last_clean":"2024-04-26T20:32:14.127842+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:32:14.127842+0000","last_undegraded":"2024-04-26T20:32:14.127842+0000","last_fullsized":"2024-04-26T20:32:14.127842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:32:16.022 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:32:16.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:32:16.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.236+0000 7f212e200640 1 Processor -- start 2024-04-26T20:32:16.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.236+0000 7f212e200640 1 -- start start 2024-04-26T20:32:16.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212e200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212e200640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f21280afe30 0x7f2128162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212e200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2128145710 con 0x7f21281628e0 2024-04-26T20:32:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212e200640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2128163280 con 0x7f21280afe30 2024-04-26T20:32:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212e200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2128073dc0 con 0x7f2128145330 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212ce00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212ce00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:35060/0 (socket says 172.21.15.43:35060) 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212ce00640 1 -- 172.21.15.43:0/2746085847 learned_addr learned my addr 172.21.15.43:0/2746085847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.238+0000 7f212d800640 1 --2- 172.21.15.43:0/2746085847 >> v2:172.21.15.164:3300/0 conn(0x7f21280afe30 0x7f2128162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.239+0000 7f212ce00640 1 -- 172.21.15.43:0/2746085847 >> v2:172.21.15.164:3300/0 conn(0x7f21280afe30 msgr2=0x7f2128162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:16.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.239+0000 7f212ce00640 1 --2- 172.21.15.43:0/2746085847 >> v2:172.21.15.164:3300/0 conn(0x7f21280afe30 0x7f2128162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.239+0000 7f212ce00640 1 -- 172.21.15.43:0/2746085847 >> v1:172.21.15.164:6789/0 conn(0x7f21281628e0 legacy=0x7f21280af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.239+0000 7f212ce00640 1 -- 172.21.15.43:0/2746085847 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280ae960 con 0x7f2128145330 2024-04-26T20:32:16.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.240+0000 7f212ce00640 1 --2- 172.21.15.43:0/2746085847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2118009870 tx=0x7f211802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f380d721d15861bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f2127e00640 1 -- 172.21.15.43:0/2746085847 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118037070 con 0x7f2128145330 2024-04-26T20:32:16.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f2127e00640 1 -- 172.21.15.43:0/2746085847 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2118032440 con 0x7f2128145330 2024-04-26T20:32:16.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f2127e00640 1 -- 172.21.15.43:0/2746085847 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118032810 con 0x7f2128145330 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 -- 172.21.15.43:0/2746085847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 msgr2=0x7f21280b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 --2- 172.21.15.43:0/2746085847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2118009870 tx=0x7f211802a270 comp rx=0 tx=0).stop 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 -- 172.21.15.43:0/2746085847 shutdown_connections 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 --2- 172.21.15.43:0/2746085847 >> v2:172.21.15.164:3300/0 conn(0x7f21280afe30 0x7f2128162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 --2- 172.21.15.43:0/2746085847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21280b3940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 -- 172.21.15.43:0/2746085847 >> 172.21.15.43:0/2746085847 conn(0x7f2128068090 msgr2=0x7f2128068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:16.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 -- 172.21.15.43:0/2746085847 shutdown_connections 2024-04-26T20:32:16.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.242+0000 7f212e200640 1 -- 172.21.15.43:0/2746085847 wait complete. 2024-04-26T20:32:16.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.244+0000 7f212e200640 1 Processor -- start 2024-04-26T20:32:16.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.244+0000 7f212e200640 1 -- start start 2024-04-26T20:32:16.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21281620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 0x7f212813f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f21280af200 con 0x7f21280afe30 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21280aef20 con 0x7f21281628e0 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.245+0000 7f212e200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f21280b3e80 con 0x7f2128145330 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34444/0 (socket says 172.21.15.43:34444) 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 -- 172.21.15.43:0/1377830876 learned_addr learned my addr 172.21.15.43:0/1377830876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:16.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212d800640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 0x7f212813f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f211fe00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21281620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 -- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 msgr2=0x7f21281620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21281620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 -- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 msgr2=0x7f212813f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 0x7f212813f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 -- 172.21.15.43:0/1377830876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280b0a00 con 0x7f21280afe30 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212d800640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 0x7f212813f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:16.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.246+0000 7f212ce00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2118000c00 tx=0x7f21180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.247+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118037040 con 0x7f21280afe30 2024-04-26T20:32:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.247+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2118042070 con 0x7f21280afe30 2024-04-26T20:32:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.247+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21280b0ce0 con 0x7f21280afe30 2024-04-26T20:32:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.247+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118030940 con 0x7f21280afe30 2024-04-26T20:32:16.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.247+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21280b12a0 con 0x7f21280afe30 2024-04-26T20:32:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.248+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2128069350 con 0x7f21280afe30 2024-04-26T20:32:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.249+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2118030ae0 con 0x7f21280afe30 2024-04-26T20:32:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.250+0000 7f2125600640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 0x7f210406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.250+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f21180ac1d0 con 0x7f21280afe30 2024-04-26T20:32:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.250+0000 7f211fe00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 0x7f210406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.251+0000 7f211fe00640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 0x7f210406ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f21280b5b40 tx=0x7f2110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.256+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f211807a800 con 0x7f21280afe30 2024-04-26T20:32:16.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.454+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f212815c600 con 0x7f210406c920 2024-04-26T20:32:16.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.455+0000 7f2125600640 1 -- 172.21.15.43:0/1377830876 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f21100123d0 con 0x7f210406c920 2024-04-26T20:32:16.457 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:16.457 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:32:16.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.457+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 msgr2=0x7f210406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.457+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 0x7f210406ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f21280b5b40 tx=0x7f2110009040 comp rx=0 tx=0).stop 2024-04-26T20:32:16.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.457+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 msgr2=0x7f212815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.457+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2118000c00 tx=0x7f21180046c0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 shutdown_connections 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f210406c920 0x7f210406ede0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2128145330 0x7f21281620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21281628e0 0x7f212813f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 --2- 172.21.15.43:0/1377830876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21280afe30 0x7f212815bc20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 >> 172.21.15.43:0/1377830876 conn(0x7f2128068090 msgr2=0x7f21280b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 shutdown_connections 2024-04-26T20:32:16.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.458+0000 7f212e200640 1 -- 172.21.15.43:0/1377830876 wait complete. 2024-04-26T20:32:16.478 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-04-26T20:32:14.990289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237936,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736784,"statfs":{"total":773094113280,"available":772850466816,"internally_reserved":0,"allocated":5251072,"data_stored":2955352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002215"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:32:14.127842+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:32:14.127842+0000","last_peered":"2024-04-26T20:32:14.127842+0000","last_clean":"2024-04-26T20:32:14.127842+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:32:14.127842+0000","last_undegraded":"2024-04-26T20:32:14.127842+0000","last_fullsized":"2024-04-26T20:32:14.127842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:32:16.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:32:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.678+0000 7f02b9800640 1 Processor -- start 2024-04-26T20:32:16.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.679+0000 7f02b9800640 1 -- start start 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.680+0000 7f02b9800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02b4072b70 0x7f02b40d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.680+0000 7f02b9800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f02b40d1060 0x7f02b406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.680+0000 7f02b9800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b4146ba0 con 0x7f02b41467c0 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.680+0000 7f02b9800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f02b4073dd0 con 0x7f02b4072b70 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.680+0000 7f02b9800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f02b40731c0 con 0x7f02b40d1060 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02b4072b70 0x7f02b40d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02b4072b70 0x7f02b40d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34456/0 (socket says 172.21.15.43:34456) 2024-04-26T20:32:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 -- 172.21.15.43:0/2961032354 learned_addr learned my addr 172.21.15.43:0/2961032354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:16.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 -- 172.21.15.43:0/2961032354 >> v2:172.21.15.164:3300/0 conn(0x7f02b40d1060 msgr2=0x7f02b406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:16.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 --2- 172.21.15.43:0/2961032354 >> v2:172.21.15.164:3300/0 conn(0x7f02b40d1060 0x7f02b406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 -- 172.21.15.43:0/2961032354 >> v1:172.21.15.164:6789/0 conn(0x7f02b41467c0 legacy=0x7f02b40b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:16.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.681+0000 7f02b2000640 1 -- 172.21.15.43:0/2961032354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b40734a0 con 0x7f02b4072b70 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.682+0000 7f02b2000640 1 --2- 172.21.15.43:0/2961032354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b4072b70 0x7f02b40d0b20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02b40720f0 tx=0x7f02a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f94d8ed0c0a0844 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b1600640 1 -- 172.21.15.43:0/2961032354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0037070 con 0x7f02b4072b70 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b1600640 1 -- 172.21.15.43:0/2961032354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a0032440 con 0x7f02b4072b70 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b1600640 1 -- 172.21.15.43:0/2961032354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0032810 con 0x7f02b4072b70 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 -- 172.21.15.43:0/2961032354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b4072b70 msgr2=0x7f02b40d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 --2- 172.21.15.43:0/2961032354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b4072b70 0x7f02b40d0b20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02b40720f0 tx=0x7f02a002a230 comp rx=0 tx=0).stop 2024-04-26T20:32:16.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 -- 172.21.15.43:0/2961032354 shutdown_connections 2024-04-26T20:32:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 --2- 172.21.15.43:0/2961032354 >> v2:172.21.15.164:3300/0 conn(0x7f02b40d1060 0x7f02b406dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 --2- 172.21.15.43:0/2961032354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b4072b70 0x7f02b40d0b20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.683+0000 7f02b9800640 1 -- 172.21.15.43:0/2961032354 >> 172.21.15.43:0/2961032354 conn(0x7f02b40b2900 msgr2=0x7f02b40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.684+0000 7f02b9800640 1 -- 172.21.15.43:0/2961032354 shutdown_connections 2024-04-26T20:32:16.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.684+0000 7f02b9800640 1 -- 172.21.15.43:0/2961032354 wait complete. 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.685+0000 7f02b9800640 1 Processor -- start 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.685+0000 7f02b9800640 1 -- start start 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 0x7f02b413d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02b40d1060 0x7f02b41f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f02b406e530 con 0x7f02b41467c0 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4073a60 con 0x7f02b40d1060 2024-04-26T20:32:16.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.686+0000 7f02b9800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f02b406e6b0 con 0x7f02b4072b70 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34464/0 (socket says 172.21.15.43:34464) 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 -- 172.21.15.43:0/376057991 learned_addr learned my addr 172.21.15.43:0/376057991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b2a00640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 0x7f02b413d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 -- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 msgr2=0x7f02b413d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 0x7f02b413d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b2000640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02b40d1060 0x7f02b41f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 -- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02b40d1060 msgr2=0x7f02b41f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02b40d1060 0x7f02b41f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b3400640 1 -- 172.21.15.43:0/376057991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b413fa60 con 0x7f02b41467c0 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.687+0000 7f02b2a00640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 0x7f02b413d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02b3400640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00b9a0 tx=0x7f02ac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac019070 con 0x7f02b41467c0 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ac00cbc0 con 0x7f02b41467c0 2024-04-26T20:32:16.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac014440 con 0x7f02b41467c0 2024-04-26T20:32:16.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b41f6c60 con 0x7f02b41467c0 2024-04-26T20:32:16.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.688+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b413fd20 con 0x7f02b41467c0 2024-04-26T20:32:16.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.689+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b4069350 con 0x7f02b41467c0 2024-04-26T20:32:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.690+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02ac01c070 con 0x7f02b41467c0 2024-04-26T20:32:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.691+0000 7f02aaa00640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 0x7f029406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.691+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f02ac08b780 con 0x7f02b41467c0 2024-04-26T20:32:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.691+0000 7f02b2a00640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 0x7f029406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:16.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.692+0000 7f02b2a00640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 0x7f029406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02a400a810 tx=0x7f02a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:16.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.696+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02ac059d10 con 0x7f02b41467c0 2024-04-26T20:32:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.903+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02b413f6c0 con 0x7f02b41467c0 2024-04-26T20:32:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.905+0000 7f02aaa00640 1 -- 172.21.15.43:0/376057991 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13265 (secure 0 0 0) 0x7f02ac059610 con 0x7f02b41467c0 2024-04-26T20:32:16.907 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:16.907 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":43,"fsid":"6a354559-7839-4522-9cd7-4f1548bb621f","created":"2024-04-26T20:25:37.610685+0000","modified":"2024-04-26T20:32:12.086269+0000","last_up_change":"2024-04-26T20:25:49.633232+0000","last_in_change":"2024-04-26T20:25:40.967754+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:25:50.893409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfca3953-6286-4ae6-8653-63d4ec362060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6801","nonce":1201466725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6803","nonce":1201466725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6807","nonce":1201466725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1201466725},{"type":"v1","addr":"172.21.15.43:6805","nonce":1201466725}]},"public_addr":"172.21.15.43:6801/1201466725","cluster_addr":"172.21.15.43:6803/1201466725","heartbeat_back_addr":"172.21.15.43:6807/1201466725","heartbeat_front_addr":"172.21.15.43:6805/1201466725","state":["exists","up"]},{"osd":1,"uuid":"b828f235-eff0-4dac-9fd9-41ea4af4465f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6809","nonce":299956676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6811","nonce":299956676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6815","nonce":299956676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":299956676},{"type":"v1","addr":"172.21.15.43:6813","nonce":299956676}]},"public_addr":"172.21.15.43:6809/299956676","cluster_addr":"172.21.15.43:6811/299956676","heartbeat_back_addr":"172.21.15.43:6815/299956676","heartbeat_front_addr":"172.21.15.43:6813/299956676","state":["exists","up"]},{"osd":2,"uuid":"1de917eb-de32-46d1-aa44-506a7a1619a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6825","nonce":637915066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6827","nonce":637915066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6831","nonce":637915066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":637915066},{"type":"v1","addr":"172.21.15.43:6829","nonce":637915066}]},"public_addr":"172.21.15.43:6825/637915066","cluster_addr":"172.21.15.43:6827/637915066","heartbeat_back_addr":"172.21.15.43:6831/637915066","heartbeat_front_addr":"172.21.15.43:6829/637915066","state":["exists","up"]},{"osd":3,"uuid":"f9fc22c8-e0be-4eaf-8351-157dcfd186e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6817","nonce":429853617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6819","nonce":429853617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6823","nonce":429853617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":429853617},{"type":"v1","addr":"172.21.15.43:6821","nonce":429853617}]},"public_addr":"172.21.15.43:6817/429853617","cluster_addr":"172.21.15.43:6819/429853617","heartbeat_back_addr":"172.21.15.43:6823/429853617","heartbeat_front_addr":"172.21.15.43:6821/429853617","state":["exists","up"]},{"osd":4,"uuid":"8d25a30b-4f74-4152-a9d9-765b60a96f2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6809","nonce":2254882506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6811","nonce":2254882506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6815","nonce":2254882506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2254882506},{"type":"v1","addr":"172.21.15.164:6813","nonce":2254882506}]},"public_addr":"172.21.15.164:6809/2254882506","cluster_addr":"172.21.15.164:6811/2254882506","heartbeat_back_addr":"172.21.15.164:6815/2254882506","heartbeat_front_addr":"172.21.15.164:6813/2254882506","state":["exists","up"]},{"osd":5,"uuid":"a14b82be-18ac-4729-b317-820a65803caa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6801","nonce":2479038766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6803","nonce":2479038766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6807","nonce":2479038766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":2479038766},{"type":"v1","addr":"172.21.15.164:6805","nonce":2479038766}]},"public_addr":"172.21.15.164:6801/2479038766","cluster_addr":"172.21.15.164:6803/2479038766","heartbeat_back_addr":"172.21.15.164:6807/2479038766","heartbeat_front_addr":"172.21.15.164:6805/2479038766","state":["exists","up"]},{"osd":6,"uuid":"decfbdf4-6d09-495f-81a4-0935e7bae1b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6817","nonce":3085699515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6819","nonce":3085699515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6828","nonce":3085699515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":3085699515},{"type":"v1","addr":"172.21.15.164:6823","nonce":3085699515}]},"public_addr":"172.21.15.164:6817/3085699515","cluster_addr":"172.21.15.164:6819/3085699515","heartbeat_back_addr":"172.21.15.164:6828/3085699515","heartbeat_front_addr":"172.21.15.164:6823/3085699515","state":["exists","up"]},{"osd":7,"uuid":"7bfe1def-9f02-4de8-b46f-2aeac197503f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6822","nonce":3685734532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6825","nonce":3685734532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6831","nonce":3685734532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6827","nonce":3685734532},{"type":"v1","addr":"172.21.15.164:6829","nonce":3685734532}]},"public_addr":"172.21.15.164:6822/3685734532","cluster_addr":"172.21.15.164:6825/3685734532","heartbeat_back_addr":"172.21.15.164:6831/3685734532","heartbeat_front_addr":"172.21.15.164:6829/3685734532","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.367620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.327958+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.136857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.547134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.269120+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.322349+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.142511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:25:46.355302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6837/37476793":"2024-04-27T20:32:09.075506+0000","172.21.15.43:6835/37476793":"2024-04-27T20:32:09.075506+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:32:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 msgr2=0x7f029406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 0x7f029406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02a400a810 tx=0x7f02a4008040 comp rx=0 tx=0).stop 2024-04-26T20:32:16.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 msgr2=0x7f02b415e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00b9a0 tx=0x7f02ac00be60 comp rx=0 tx=0).stop 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 shutdown_connections 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f029406c8f0 0x7f029406edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02b41467c0 0x7f02b415e150 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f02b40d1060 0x7f02b41f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 --2- 172.21.15.43:0/376057991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f02b4072b70 0x7f02b413d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.907+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 >> 172.21.15.43:0/376057991 conn(0x7f02b40b2900 msgr2=0x7f02b40b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.908+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 shutdown_connections 2024-04-26T20:32:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:16.908+0000 7f02b9800640 1 -- 172.21.15.43:0/376057991 wait complete. 2024-04-26T20:32:17.928 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T20:32:17.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:18.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.154+0000 7f1422a00640 1 Processor -- start 2024-04-26T20:32:18.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.154+0000 7f1422a00640 1 -- start start 2024-04-26T20:32:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.156+0000 7f1422a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.156+0000 7f1422a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f141c15c450 0x7f141c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1422a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141c070150 con 0x7f141c0b46d0 2024-04-26T20:32:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1422a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f141c0b3c10 con 0x7f141c15c450 2024-04-26T20:32:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1422a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f141c15d180 con 0x7f141c06f830 2024-04-26T20:32:18.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1413e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f141c15c450 0x7f141c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1420a00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f141c0b46d0 0x7f141c0b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59220/0 (socket says 172.21.15.43:59220) 2024-04-26T20:32:18.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1420a00640 1 -- 172.21.15.43:0/1366035938 learned_addr learned my addr 172.21.15.43:0/1366035938 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:18.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f141be00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1413e00640 1 -- 172.21.15.43:0/1366035938 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 msgr2=0x7f141c06fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:18.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1413e00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c06fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1413e00640 1 -- 172.21.15.43:0/1366035938 >> v1:172.21.15.43:6789/0 conn(0x7f141c0b46d0 legacy=0x7f141c0b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.157+0000 7f1413e00640 1 -- 172.21.15.43:0/1366035938 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c0b3ed0 con 0x7f141c15c450 2024-04-26T20:32:18.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.159+0000 7f1413e00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c15c8d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f14040097a0 tx=0x7f140402c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d030455161702419 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.159+0000 7f141b400640 1 -- 172.21.15.43:0/1366035938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404039070 con 0x7f141c15c450 2024-04-26T20:32:18.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.159+0000 7f141b400640 1 -- 172.21.15.43:0/1366035938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140402ce50 con 0x7f141c15c450 2024-04-26T20:32:18.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.159+0000 7f141b400640 1 -- 172.21.15.43:0/1366035938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404034860 con 0x7f141c15c450 2024-04-26T20:32:18.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 -- 172.21.15.43:0/1366035938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 msgr2=0x7f141c15c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c15c8d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f14040097a0 tx=0x7f140402c300 comp rx=0 tx=0).stop 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 -- 172.21.15.43:0/1366035938 shutdown_connections 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c15c8d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 --2- 172.21.15.43:0/1366035938 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 -- 172.21.15.43:0/1366035938 >> 172.21.15.43:0/1366035938 conn(0x7f141c0b2790 msgr2=0x7f141c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 -- 172.21.15.43:0/1366035938 shutdown_connections 2024-04-26T20:32:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.160+0000 7f1422a00640 1 -- 172.21.15.43:0/1366035938 wait complete. 2024-04-26T20:32:18.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.162+0000 7f1422a00640 1 Processor -- start 2024-04-26T20:32:18.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.162+0000 7f1422a00640 1 -- start start 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f141c0b46d0 0x7f141c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c1f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f141c0738a0 con 0x7f141c15c450 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f141c073720 con 0x7f141c0b46d0 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1422a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f141c0b5270 con 0x7f141c06f830 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.163+0000 7f1420a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c1f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c1f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34488/0 (socket says 172.21.15.43:34488) 2024-04-26T20:32:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 -- 172.21.15.43:0/514569827 learned_addr learned my addr 172.21.15.43:0/514569827 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f141be00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1413e00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f141c0b46d0 0x7f141c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 -- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 msgr2=0x7f141c148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f141c06f830 0x7f141c148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 -- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f141c0b46d0 msgr2=0x7f141c159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f141c0b46d0 0x7f141c159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c148d40 con 0x7f141c15c450 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1413e00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f141c0b46d0 0x7f141c159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.164+0000 7f1420a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c1f5210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f141400caa0 tx=0x7f141400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.165+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414017070 con 0x7f141c15c450 2024-04-26T20:32:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.165+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141400ac00 con 0x7f141c15c450 2024-04-26T20:32:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.165+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c0cf890 con 0x7f141c15c450 2024-04-26T20:32:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.165+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414012400 con 0x7f141c15c450 2024-04-26T20:32:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.165+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c0cfe50 con 0x7f141c15c450 2024-04-26T20:32:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.166+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f141c0cf640 con 0x7f141c15c450 2024-04-26T20:32:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.167+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f141401a070 con 0x7f141c15c450 2024-04-26T20:32:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1418c00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f13f806c8f0 0x7f13f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f141408b6b0 con 0x7f141c15c450 2024-04-26T20:32:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f141be00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f13f806c8f0 0x7f13f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1418c00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7f13f8072d90 0x7f13f80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13f80758a0 con 0x7f13f8072d90 2024-04-26T20:32:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1414092050 con 0x7f141c15c450 2024-04-26T20:32:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.169+0000 7f1413e00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7f13f8072d90 0x7f13f80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.170+0000 7f141be00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f13f806c8f0 0x7f13f806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f140c006fd0 tx=0x7f140c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.170+0000 7f1413e00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7f13f8072d90 0x7f13f80751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.171+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== osd.0 v2:172.21.15.43:6800/1201466725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13f80758a0 con 0x7f13f8072d90 2024-04-26T20:32:18.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.202+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 --> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f141c069350 con 0x7f13f8072d90 2024-04-26T20:32:18.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1418c00640 1 -- 172.21.15.43:0/514569827 <== osd.0 v2:172.21.15.43:6800/1201466725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f141c069350 con 0x7f13f8072d90 2024-04-26T20:32:18.203 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:18.203 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:18.203 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7f13f8072d90 msgr2=0x7f13f80751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6800/1201466725,v1:172.21.15.43:6801/1201466725] conn(0x7f13f8072d90 0x7f13f80751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f13f806c8f0 msgr2=0x7f13f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f13f806c8f0 0x7f13f806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f140c006fd0 tx=0x7f140c008040 comp rx=0 tx=0).stop 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 msgr2=0x7f141c1f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1422a00640 1 --2- 172.21.15.43:0/514569827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f141c15c450 0x7f141c1f5210 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f141400caa0 tx=0x7f141400cf60 comp rx=0 tx=0).stop 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.203+0000 7f1420a00640 1 -- 172.21.15.43:0/514569827 reap_dead start 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.204+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 shutdown_connections 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.204+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 >> 172.21.15.43:0/514569827 conn(0x7f141c0b2790 msgr2=0x7f141c074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.204+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 shutdown_connections 2024-04-26T20:32:18.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.204+0000 7f1422a00640 1 -- 172.21.15.43:0/514569827 wait complete. 2024-04-26T20:32:18.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T20:32:18.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.423+0000 7f5a9fe00640 1 Processor -- start 2024-04-26T20:32:18.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.423+0000 7f5a9fe00640 1 -- start start 2024-04-26T20:32:18.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a9fe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5aa015f170 0x7f5aa00d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a9fe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00d0f10 con 0x7f5aa00d5850 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a9fe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa00d0c30 con 0x7f5aa00d3870 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a9fe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa00b42a0 con 0x7f5aa015f170 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a9ea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5aa015f170 0x7f5aa00d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a97e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5aa00d5850 0x7f5aa00d3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59230/0 (socket says 172.21.15.43:59230) 2024-04-26T20:32:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.424+0000 7f5a97e00640 1 -- 172.21.15.43:0/613412537 learned_addr learned my addr 172.21.15.43:0/613412537 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:18.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.425+0000 7f5a9ea00640 1 -- 172.21.15.43:0/613412537 >> v1:172.21.15.164:6789/0 conn(0x7f5aa00d3870 legacy=0x7f5aa00b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:18.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.425+0000 7f5a9ea00640 1 -- 172.21.15.43:0/613412537 >> v1:172.21.15.43:6789/0 conn(0x7f5aa00d5850 legacy=0x7f5aa00d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.425+0000 7f5a9ea00640 1 -- 172.21.15.43:0/613412537 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa00b4580 con 0x7f5aa015f170 2024-04-26T20:32:18.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.425+0000 7f5a9e000640 1 -- 172.21.15.43:0/613412537 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239588412 0 0) 0x7f5aa00d0f10 con 0x7f5aa00d5850 2024-04-26T20:32:18.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.426+0000 7f5a9ea00640 1 --2- 172.21.15.43:0/613412537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa015f170 0x7f5aa00d2a80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5a900097a0 tx=0x7f5a9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b70bd80bf903880 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.426+0000 7f5a9e000640 1 -- 172.21.15.43:0/613412537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90037070 con 0x7f5aa015f170 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.426+0000 7f5a9e000640 1 -- 172.21.15.43:0/613412537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a90032440 con 0x7f5aa015f170 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.426+0000 7f5a9e000640 1 -- 172.21.15.43:0/613412537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90032810 con 0x7f5aa015f170 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 -- 172.21.15.43:0/613412537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa015f170 msgr2=0x7f5aa00d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/613412537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa015f170 0x7f5aa00d2a80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5a900097a0 tx=0x7f5a9002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 -- 172.21.15.43:0/613412537 shutdown_connections 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/613412537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa015f170 0x7f5aa00d2a80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 -- 172.21.15.43:0/613412537 >> 172.21.15.43:0/613412537 conn(0x7f5aa00d4ca0 msgr2=0x7f5aa00d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 -- 172.21.15.43:0/613412537 shutdown_connections 2024-04-26T20:32:18.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.427+0000 7f5a9fe00640 1 -- 172.21.15.43:0/613412537 wait complete. 2024-04-26T20:32:18.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.429+0000 7f5a9fe00640 1 Processor -- start 2024-04-26T20:32:18.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.429+0000 7f5a9fe00640 1 -- start start 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5aa00d3870 0x7f5aa00cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa015f170 0x7f5aa01f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa00d0850 con 0x7f5aa00d5850 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa015f550 con 0x7f5aa015f170 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a9fe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa00d09d0 con 0x7f5aa00d3870 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34518/0 (socket says 172.21.15.43:34518) 2024-04-26T20:32:18.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 -- 172.21.15.43:0/840829563 learned_addr learned my addr 172.21.15.43:0/840829563 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 -- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5aa00d3870 msgr2=0x7f5aa00cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5aa00d3870 0x7f5aa00cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 -- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa015f170 msgr2=0x7f5aa01f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa015f170 0x7f5aa01f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.430+0000 7f5a97e00640 1 -- 172.21.15.43:0/840829563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa00cc400 con 0x7f5aa00d5850 2024-04-26T20:32:18.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a97e00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c470 tx=0x7f5a8800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88018070 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a880074e0 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88004a20 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa00cc6e0 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.431+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa00ccca0 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.432+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa0069a60 con 0x7f5aa00d5850 2024-04-26T20:32:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.433+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a88004bc0 con 0x7f5aa00d5850 2024-04-26T20:32:18.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.434+0000 7f5a96a00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 0x7f5a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.434+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f5a8808ae80 con 0x7f5aa00d5850 2024-04-26T20:32:18.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.435+0000 7f5a9ea00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 0x7f5a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.435+0000 7f5a9ea00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 0x7f5a7c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a90034000 tx=0x7f5a9002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.439+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a880592f0 con 0x7f5aa00d5850 2024-04-26T20:32:18.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.641+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f5aa015e730 con 0x7f5a7c06c8f0 2024-04-26T20:32:18.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.642+0000 7f5a96a00640 1 -- 172.21.15.43:0/840829563 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5aa015e730 con 0x7f5a7c06c8f0 2024-04-26T20:32:18.642 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T20:32:18.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 msgr2=0x7f5a7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 0x7f5a7c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a90034000 tx=0x7f5a9002c000 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 msgr2=0x7f5aa015cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c470 tx=0x7f5a8800c930 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 shutdown_connections 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5a7c06c8f0 0x7f5a7c06edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa015f170 0x7f5aa01f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5aa00d5850 0x7f5aa015cb40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 --2- 172.21.15.43:0/840829563 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5aa00d3870 0x7f5aa00cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 >> 172.21.15.43:0/840829563 conn(0x7f5aa00d4ca0 msgr2=0x7f5aa00b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 shutdown_connections 2024-04-26T20:32:18.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.644+0000 7f5a9fe00640 1 -- 172.21.15.43:0/840829563 wait complete. 2024-04-26T20:32:18.661 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T20:32:18.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.852+0000 7f93f0c00640 1 Processor -- start 2024-04-26T20:32:18.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.852+0000 7f93f0c00640 1 -- start start 2024-04-26T20:32:18.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93f0c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f93ec073940 0x7f93ec0d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93f0c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec0b1480 con 0x7f93ec0b10a0 2024-04-26T20:32:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93f0c00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec073e00 con 0x7f93ec0b6b40 2024-04-26T20:32:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93f0c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec06f300 con 0x7f93ec073940 2024-04-26T20:32:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93eb400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f93ec0b10a0 0x7f93ec070510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59238/0 (socket says 172.21.15.43:59238) 2024-04-26T20:32:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93eb400640 1 -- 172.21.15.43:0/771604034 learned_addr learned my addr 172.21.15.43:0/771604034 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.854+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743585248 0 0) 0x7f93ec0b1480 con 0x7f93ec0b10a0 2024-04-26T20:32:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ebe00640 1 --2- 172.21.15.43:0/771604034 >> v2:172.21.15.164:3300/0 conn(0x7f93ec073940 0x7f93ec0d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d0003520 con 0x7f93ec0b10a0 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3719271651 0 0) 0x7f93d0003520 con 0x7f93ec0b10a0 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 >> v2:172.21.15.164:3300/0 conn(0x7f93ec073940 msgr2=0x7f93ec0d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 --2- 172.21.15.43:0/771604034 >> v2:172.21.15.164:3300/0 conn(0x7f93ec073940 0x7f93ec0d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 >> v1:172.21.15.164:6789/0 conn(0x7f93ec0b6b40 legacy=0x7f93ec06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec0b4c40 con 0x7f93ec0b10a0 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ebe00640 1 --2- 172.21.15.43:0/771604034 >> v2:172.21.15.164:3300/0 conn(0x7f93ec073940 0x7f93ec0d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f93dc001670 con 0x7f93ec0b10a0 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 >> v1:172.21.15.43:6789/0 conn(0x7f93ec0b10a0 legacy=0x7f93ec070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93d00039b0 0x7f93d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93d00053c0 0x7f93d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec06f300 con 0x7f93d0004800 2024-04-26T20:32:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93d000c250 con 0x7f93d00053c0 2024-04-26T20:32:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.855+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93d000c450 con 0x7f93d00039b0 2024-04-26T20:32:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eb400640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93d00039b0 0x7f93d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93ebe00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93d00053c0 0x7f93d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 -- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93d00039b0 msgr2=0x7f93d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93d00039b0 0x7f93d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 -- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93d00053c0 msgr2=0x7f93d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93d00053c0 0x7f93d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93eaa00640 1 -- 172.21.15.43:0/771604034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d0003520 con 0x7f93d0004800 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93ebe00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93d00053c0 0x7f93d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.856+0000 7f93ebe00640 1 -- 172.21.15.43:0/771604034 reap_dead start 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.857+0000 7f93eaa00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93d8001750 tx=0x7f93d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3383ef0067710fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.857+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8001370 con 0x7f93d0004800 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.857+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d80052d0 con 0x7f93d0004800 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.857+0000 7f93ea000640 1 -- 172.21.15.43:0/771604034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d802abc0 con 0x7f93d0004800 2024-04-26T20:32:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 -- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 msgr2=0x7f93d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f93d8001750 tx=0x7f93d0005830 comp rx=0 tx=0).stop 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 -- 172.21.15.43:0/771604034 shutdown_connections 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 --2- 172.21.15.43:0/771604034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93d0004c80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 -- 172.21.15.43:0/771604034 >> 172.21.15.43:0/771604034 conn(0x7f93ec068090 msgr2=0x7f93ec068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 -- 172.21.15.43:0/771604034 shutdown_connections 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.858+0000 7f93f0c00640 1 -- 172.21.15.43:0/771604034 wait complete. 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.860+0000 7f93f0c00640 1 Processor -- start 2024-04-26T20:32:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.860+0000 7f93f0c00640 1 -- start start 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93ec13f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1445e0 0x7f93ec144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93ec15fd00 0x7f93ec160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec06ebf0 con 0x7f93d0004800 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0b65b0 con 0x7f93ec1445e0 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93f0c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec06ed70 con 0x7f93ec15fd00 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93ec13f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93ec13f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34530/0 (socket says 172.21.15.43:34530) 2024-04-26T20:32:18.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.861+0000 7f93eb400640 1 -- 172.21.15.43:0/2494311184 learned_addr learned my addr 172.21.15.43:0/2494311184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93ebe00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93ec15fd00 0x7f93ec160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eaa00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1445e0 0x7f93ec144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 -- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93ec15fd00 msgr2=0x7f93ec160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93ec15fd00 0x7f93ec160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 -- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1445e0 msgr2=0x7f93ec144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1445e0 0x7f93ec144a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec160a60 con 0x7f93d0004800 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eaa00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1445e0 0x7f93ec144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93eb400640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93ec13f200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e700 tx=0x7f93dc00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00c5c0 con 0x7f93d0004800 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc018070 con 0x7f93d0004800 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00c9a0 con 0x7f93d0004800 2024-04-26T20:32:18.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1f51b0 con 0x7f93d0004800 2024-04-26T20:32:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.862+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec13f740 con 0x7f93d0004800 2024-04-26T20:32:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.864+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93ec160d40 con 0x7f93d0004800 2024-04-26T20:32:18.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.865+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93dc00cb40 con 0x7f93d0004800 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.865+0000 7f93e3400640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f93cc06c920 0x7f93cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f93dc08b400 con 0x7f93d0004800 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93e3400640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f93cc072dc0 0x7f93cc075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93eaa00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f93cc06c920 0x7f93cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93cc0758d0 con 0x7f93cc072dc0 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f93dc092050 con 0x7f93d0004800 2024-04-26T20:32:18.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.866+0000 7f93ebe00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f93cc072dc0 0x7f93cc075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.867+0000 7f93eaa00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f93cc06c920 0x7f93cc06ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f93d8032000 tx=0x7f93d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.867+0000 7f93ebe00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f93cc072dc0 0x7f93cc075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.867+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== osd.1 v2:172.21.15.43:6808/299956676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93cc0758d0 con 0x7f93cc072dc0 2024-04-26T20:32:18.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.896+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 --> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93ec1f4fd0 con 0x7f93cc072dc0 2024-04-26T20:32:18.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.896+0000 7f93e3400640 1 -- 172.21.15.43:0/2494311184 <== osd.1 v2:172.21.15.43:6808/299956676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93ec1f4fd0 con 0x7f93cc072dc0 2024-04-26T20:32:18.897 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:18.897 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:18.897 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:18.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f93cc072dc0 msgr2=0x7f93cc075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6808/299956676,v1:172.21.15.43:6809/299956676] conn(0x7f93cc072dc0 0x7f93cc075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f93cc06c920 msgr2=0x7f93cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f93cc06c920 0x7f93cc06ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f93d8032000 tx=0x7f93d802e000 comp rx=0 tx=0).stop 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 msgr2=0x7f93ec13f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93f0c00640 1 --2- 172.21.15.43:0/2494311184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d0004800 0x7f93ec13f200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e700 tx=0x7f93dc00eb00 comp rx=0 tx=0).stop 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.897+0000 7f93ebe00640 1 -- 172.21.15.43:0/2494311184 reap_dead start 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.898+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 shutdown_connections 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.898+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 >> 172.21.15.43:0/2494311184 conn(0x7f93ec068090 msgr2=0x7f93ec0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.898+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 shutdown_connections 2024-04-26T20:32:18.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:18.898+0000 7f93f0c00640 1 -- 172.21.15.43:0/2494311184 wait complete. 2024-04-26T20:32:18.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.120+0000 7f22f2c00640 1 Processor -- start 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.120+0000 7f22f2c00640 1 -- start start 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22f2c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22ec15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22f2c00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22ec15ef00 0x7f22ec0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22f2c00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec0c8830 con 0x7f22ec0b71a0 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22f2c00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec0d3cb0 con 0x7f22ec15ef00 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22f2c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec0c8af0 con 0x7f22ec0b7580 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22ebe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f22ec0b71a0 0x7f22ec0d34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59252/0 (socket says 172.21.15.43:59252) 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.121+0000 7f22ebe00640 1 -- 172.21.15.43:0/3710909880 learned_addr learned my addr 172.21.15.43:0/3710909880 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22f0c00640 1 --2- 172.21.15.43:0/3710909880 >> v2:172.21.15.164:3300/0 conn(0x7f22ec15ef00 0x7f22ec0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991376577 0 0) 0x7f22ec0c8830 con 0x7f22ec0b71a0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d0003520 con 0x7f22ec0b71a0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22e3400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22ec15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1562297550 0 0) 0x7f22d0003520 con 0x7f22ec0b71a0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 msgr2=0x7f22ec15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22ec15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 >> v2:172.21.15.164:3300/0 conn(0x7f22ec15ef00 msgr2=0x7f22ec0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 --2- 172.21.15.43:0/3710909880 >> v2:172.21.15.164:3300/0 conn(0x7f22ec15ef00 0x7f22ec0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0c8dd0 con 0x7f22ec0b71a0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f22dc001670 con 0x7f22ec0b71a0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 >> v1:172.21.15.43:6789/0 conn(0x7f22ec0b71a0 legacy=0x7f22ec0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22d00039b0 0x7f22d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d0004d40 0x7f22d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.122+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0d3cb0 con 0x7f22d00039b0 2024-04-26T20:32:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0c8af0 con 0x7f22d0004d40 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22d000ba70 con 0x7f22ec0b7580 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22d00039b0 0x7f22d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22e3400640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22f0c00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d0004d40 0x7f22d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 -- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 msgr2=0x7f22d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec0b7580 0x7f22d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 -- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d0004d40 msgr2=0x7f22d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d0004d40 0x7f22d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.123+0000 7f22ebe00640 1 -- 172.21.15.43:0/3710909880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d0003520 con 0x7f22d00039b0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.124+0000 7f22ebe00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22d00039b0 0x7f22d0003dd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c210 tx=0x7f22d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1cd3a7a0a5ee1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007d70 con 0x7f22d00039b0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22dc005ad0 con 0x7f22d00039b0 2024-04-26T20:32:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22eb400640 1 -- 172.21.15.43:0/3710909880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007070 con 0x7f22d00039b0 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 -- 172.21.15.43:0/3710909880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22d00039b0 msgr2=0x7f22d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 --2- 172.21.15.43:0/3710909880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22d00039b0 0x7f22d0003dd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c210 tx=0x7f22d0005830 comp rx=0 tx=0).stop 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f0c00640 1 -- 172.21.15.43:0/3710909880 reap_dead start 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 -- 172.21.15.43:0/3710909880 shutdown_connections 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 -- 172.21.15.43:0/3710909880 >> 172.21.15.43:0/3710909880 conn(0x7f22ec0d4d70 msgr2=0x7f22ec0d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 -- 172.21.15.43:0/3710909880 shutdown_connections 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.125+0000 7f22f2c00640 1 -- 172.21.15.43:0/3710909880 wait complete. 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.127+0000 7f22f2c00640 1 Processor -- start 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.127+0000 7f22f2c00640 1 -- start start 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00039b0 0x7f22ec0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec15b990 0x7f22ec15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec15f530 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec15f3b0 con 0x7f22d00039b0 2024-04-26T20:32:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.128+0000 7f22f2c00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec15f6b0 con 0x7f22ec15b990 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34542/0 (socket says 172.21.15.43:34542) 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 -- 172.21.15.43:0/482957992 learned_addr learned my addr 172.21.15.43:0/482957992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22ebe00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00039b0 0x7f22ec0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22e3400640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec15b990 0x7f22ec15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 -- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec15b990 msgr2=0x7f22ec15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec15b990 0x7f22ec15be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 -- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00039b0 msgr2=0x7f22ec0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00039b0 0x7f22ec0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.129+0000 7f22f0c00640 1 -- 172.21.15.43:0/482957992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0b99a0 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.130+0000 7f22f0c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0bae00 tx=0x7f22e4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.131+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4019020 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.131+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e4003610 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.131+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4003a00 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.131+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec15fa40 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.132+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec160000 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.133+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec069350 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.134+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22e401c020 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.134+0000 7f22e8c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.135+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f22e408bb30 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.135+0000 7f22ebe00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.136+0000 7f22ebe00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 0x7f22c806ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22dc033000 tx=0x7f22dc009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.139+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22e4023030 con 0x7f22ec1f4ea0 2024-04-26T20:32:19.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.339+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f22ec1607e0 con 0x7f22c806c920 2024-04-26T20:32:19.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.340+0000 7f22e8c00640 1 -- 172.21.15.43:0/482957992 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f22ec1607e0 con 0x7f22c806c920 2024-04-26T20:32:19.340 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T20:32:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.342+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 msgr2=0x7f22c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.342+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 0x7f22c806ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22dc033000 tx=0x7f22dc009030 comp rx=0 tx=0).stop 2024-04-26T20:32:19.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.342+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 msgr2=0x7f22ec1f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.342+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0bae00 tx=0x7f22e4011920 comp rx=0 tx=0).stop 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 shutdown_connections 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22ec15b990 0x7f22ec15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22ec1f4ea0 0x7f22ec1f5320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 --2- 172.21.15.43:0/482957992 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00039b0 0x7f22ec0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 >> 172.21.15.43:0/482957992 conn(0x7f22ec0d4d70 msgr2=0x7f22ec074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 shutdown_connections 2024-04-26T20:32:19.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.343+0000 7f22f2c00640 1 -- 172.21.15.43:0/482957992 wait complete. 2024-04-26T20:32:19.361 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T20:32:19.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.562+0000 7f84ed600640 1 Processor -- start 2024-04-26T20:32:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.563+0000 7f84ed600640 1 -- start start 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84ed600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84e815f040 0x7f84e80d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84ed600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d3a20 0x7f84e80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84ed600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e80d03f0 con 0x7f84e80d59a0 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84ed600640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f84e80d0c90 con 0x7f84e815f040 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84ed600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80d06d0 con 0x7f84e80d3a20 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84e7400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f84e80d59a0 0x7f84e80d3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59262/0 (socket says 172.21.15.43:59262) 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84e7400640 1 -- 172.21.15.43:0/2392103416 learned_addr learned my addr 172.21.15.43:0/2392103416 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.564+0000 7f84e7e00640 1 --2- 172.21.15.43:0/2392103416 >> v2:172.21.15.43:3300/0 conn(0x7f84e815f040 0x7f84e80d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e6a00640 1 -- 172.21.15.43:0/2392103416 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 152305432 0 0) 0x7f84e80d03f0 con 0x7f84e80d59a0 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e6a00640 1 -- 172.21.15.43:0/2392103416 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84cc003520 con 0x7f84e80d59a0 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e7e00640 1 -- 172.21.15.43:0/2392103416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d3a20 msgr2=0x7f84e80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e7e00640 1 --2- 172.21.15.43:0/2392103416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d3a20 0x7f84e80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e7e00640 1 -- 172.21.15.43:0/2392103416 >> v1:172.21.15.43:6789/0 conn(0x7f84e80d59a0 legacy=0x7f84e80d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.565+0000 7f84e7e00640 1 -- 172.21.15.43:0/2392103416 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e80d09b0 con 0x7f84e815f040 2024-04-26T20:32:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.567+0000 7f84e7e00640 1 --2- 172.21.15.43:0/2392103416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e815f040 0x7f84e80d2b10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa94c12e6ddd5f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.568+0000 7f84e6a00640 1 -- 172.21.15.43:0/2392103416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8037070 con 0x7f84e815f040 2024-04-26T20:32:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.568+0000 7f84e6a00640 1 -- 172.21.15.43:0/2392103416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d802ae50 con 0x7f84e815f040 2024-04-26T20:32:19.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.568+0000 7f84e6a00640 1 -- 172.21.15.43:0/2392103416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8032660 con 0x7f84e815f040 2024-04-26T20:32:19.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.568+0000 7f84ed600640 1 -- 172.21.15.43:0/2392103416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e815f040 msgr2=0x7f84e80d2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.568+0000 7f84ed600640 1 --2- 172.21.15.43:0/2392103416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e815f040 0x7f84e80d2b10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802a100 comp rx=0 tx=0).stop 2024-04-26T20:32:19.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 -- 172.21.15.43:0/2392103416 shutdown_connections 2024-04-26T20:32:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 --2- 172.21.15.43:0/2392103416 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d3a20 0x7f84e80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 --2- 172.21.15.43:0/2392103416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e815f040 0x7f84e80d2b10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 -- 172.21.15.43:0/2392103416 >> 172.21.15.43:0/2392103416 conn(0x7f84e80d4d30 msgr2=0x7f84e80d5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 -- 172.21.15.43:0/2392103416 shutdown_connections 2024-04-26T20:32:19.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.569+0000 7f84ed600640 1 -- 172.21.15.43:0/2392103416 wait complete. 2024-04-26T20:32:19.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.571+0000 7f84ed600640 1 Processor -- start 2024-04-26T20:32:19.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.571+0000 7f84ed600640 1 -- start start 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 0x7f84e815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d59a0 0x7f84e815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e815f040 0x7f84e80cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80b5220 con 0x7f84e80d3a20 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84e815f420 con 0x7f84e815f040 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.572+0000 7f84ed600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80b53a0 con 0x7f84e80d59a0 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 0x7f84e815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 0x7f84e815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34568/0 (socket says 172.21.15.43:34568) 2024-04-26T20:32:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 -- 172.21.15.43:0/174552514 learned_addr learned my addr 172.21.15.43:0/174552514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7400640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d59a0 0x7f84e815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84ecc00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e815f040 0x7f84e80cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 -- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d59a0 msgr2=0x7f84e815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f84e80d59a0 0x7f84e815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 -- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e815f040 msgr2=0x7f84e80cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e815f040 0x7f84e80cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84e7e00640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e815fed0 con 0x7f84e80d3a20 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.573+0000 7f84ecc00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84e815f040 0x7f84e80cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84e7e00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 0x7f84e815c240 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f84d80098c0 tx=0x7f84d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8042070 con 0x7f84e80d3a20 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d803e460 con 0x7f84e80d3a20 2024-04-26T20:32:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81601b0 con 0x7f84e80d3a20 2024-04-26T20:32:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8037040 con 0x7f84e80d3a20 2024-04-26T20:32:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.574+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e815c780 con 0x7f84e80d3a20 2024-04-26T20:32:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.575+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84e80b7e40 con 0x7f84e80d3a20 2024-04-26T20:32:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.576+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84d8045070 con 0x7f84e80d3a20 2024-04-26T20:32:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.577+0000 7f84dfe00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84c806c920 0x7f84c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.577+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f84d80ac770 con 0x7f84e80d3a20 2024-04-26T20:32:19.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84e7400640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84c806c920 0x7f84c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84dfe00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7f84c8072dc0 0x7f84c8075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84c80758d0 con 0x7f84c8072dc0 2024-04-26T20:32:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f84d80acc00 con 0x7f84e80d3a20 2024-04-26T20:32:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84ecc00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7f84c8072dc0 0x7f84c8075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.578+0000 7f84e7400640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84c806c920 0x7f84c806ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84e80bad00 tx=0x7f84d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.579+0000 7f84ecc00640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7f84c8072dc0 0x7f84c8075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.579+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== osd.2 v2:172.21.15.43:6824/637915066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84c80758d0 con 0x7f84c8072dc0 2024-04-26T20:32:19.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.594+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 --> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f84e8069a60 con 0x7f84c8072dc0 2024-04-26T20:32:19.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84dfe00640 1 -- 172.21.15.43:0/174552514 <== osd.2 v2:172.21.15.43:6824/637915066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f84e8069a60 con 0x7f84c8072dc0 2024-04-26T20:32:19.595 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:19.595 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:19.595 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7f84c8072dc0 msgr2=0x7f84c8075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6824/637915066,v1:172.21.15.43:6825/637915066] conn(0x7f84c8072dc0 0x7f84c8075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84c806c920 msgr2=0x7f84c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f84c806c920 0x7f84c806ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84e80bad00 tx=0x7f84d4009040 comp rx=0 tx=0).stop 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 msgr2=0x7f84e815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ed600640 1 --2- 172.21.15.43:0/174552514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84e80d3a20 0x7f84e815c240 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f84d80098c0 tx=0x7f84d80046c0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.595+0000 7f84ecc00640 1 -- 172.21.15.43:0/174552514 reap_dead start 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.596+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 shutdown_connections 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.596+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 >> 172.21.15.43:0/174552514 conn(0x7f84e80d4d30 msgr2=0x7f84e815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.596+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 shutdown_connections 2024-04-26T20:32:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.596+0000 7f84ed600640 1 -- 172.21.15.43:0/174552514 wait complete. 2024-04-26T20:32:19.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T20:32:19.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.808+0000 7f5ea2e00640 1 Processor -- start 2024-04-26T20:32:19.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.809+0000 7f5ea2e00640 1 -- start start 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.810+0000 7f5ea2e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e9c0ce480 0x7f5e9c167b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.810+0000 7f5ea2e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c072a80 con 0x7f5e9c072620 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.810+0000 7f5ea2e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c0cd560 con 0x7f5e9c0cd180 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.810+0000 7f5ea2e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5e9c072d00 con 0x7f5e9c0ce480 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.811+0000 7f5ea1000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5e9c072620 0x7f5e9c0cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59264/0 (socket says 172.21.15.43:59264) 2024-04-26T20:32:19.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.811+0000 7f5ea1000640 1 -- 172.21.15.43:0/1206934190 learned_addr learned my addr 172.21.15.43:0/1206934190 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.811+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861251704 0 0) 0x7f5e88001110 con 0x7f5e9c072620 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.811+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e84003520 con 0x7f5e9c072620 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.811+0000 7f5ea2400640 1 --2- 172.21.15.43:0/1206934190 >> v2:172.21.15.164:3300/0 conn(0x7f5e9c0ce480 0x7f5e9c167b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2276640337 0 0) 0x7f5e84003520 con 0x7f5e9c072620 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 >> v2:172.21.15.164:3300/0 conn(0x7f5e9c0ce480 msgr2=0x7f5e9c167b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 --2- 172.21.15.43:0/1206934190 >> v2:172.21.15.164:3300/0 conn(0x7f5e9c0ce480 0x7f5e9c167b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 >> v1:172.21.15.164:6789/0 conn(0x7f5e9c0cd180 legacy=0x7f5e9c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c072fe0 con 0x7f5e9c072620 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f5e88001840 con 0x7f5e9c072620 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 >> v1:172.21.15.43:6789/0 conn(0x7f5e9c072620 legacy=0x7f5e9c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e84003a00 0x7f5e84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84004800 0x7f5e84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 0x7f5e8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c072d00 con 0x7f5e84005210 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8400c160 con 0x7f5e84003a00 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8400c360 con 0x7f5e84004800 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.812+0000 7f5ea2400640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 0x7f5e8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea1000640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84004800 0x7f5e84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea1a00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e84003a00 0x7f5e84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea2400640 1 -- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84004800 msgr2=0x7f5e84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea2400640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84004800 0x7f5e84004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea2400640 1 -- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e84003a00 msgr2=0x7f5e84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea2400640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e84003a00 0x7f5e84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea2400640 1 -- 172.21.15.43:0/1206934190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e84003520 con 0x7f5e84005210 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.813+0000 7f5ea1a00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e84003a00 0x7f5e84003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:19.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.814+0000 7f5ea2400640 1 -- 172.21.15.43:0/1206934190 reap_dead start 2024-04-26T20:32:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.814+0000 7f5ea2400640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 0x7f5e8400b9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5e94007360 tx=0x7f5e84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda01677bbdb2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.814+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94009be0 con 0x7f5e84005210 2024-04-26T20:32:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.814+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e94003b90 con 0x7f5e84005210 2024-04-26T20:32:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.814+0000 7f5e9be00640 1 -- 172.21.15.43:0/1206934190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94004020 con 0x7f5e84005210 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 -- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 msgr2=0x7f5e8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 0x7f5e8400b9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5e94007360 tx=0x7f5e84005830 comp rx=0 tx=0).stop 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 -- 172.21.15.43:0/1206934190 shutdown_connections 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/1206934190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e84005210 0x7f5e8400b9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 -- 172.21.15.43:0/1206934190 >> 172.21.15.43:0/1206934190 conn(0x7f5e9c068090 msgr2=0x7f5e9c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 -- 172.21.15.43:0/1206934190 shutdown_connections 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.815+0000 7f5ea2e00640 1 -- 172.21.15.43:0/1206934190 wait complete. 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.816+0000 7f5ea2e00640 1 Processor -- start 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.817+0000 7f5ea2e00640 1 -- start start 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.817+0000 7f5ea2e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84005210 0x7f5e9c1e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.817+0000 7f5ea2e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea2e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9c144aa0 0x7f5e9c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea2e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c073da0 con 0x7f5e9c1e8110 2024-04-26T20:32:19.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea2e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c0735a0 con 0x7f5e9c144aa0 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea2e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c0cdd70 con 0x7f5e84005210 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34584/0 (socket says 172.21.15.43:34584) 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 -- 172.21.15.43:0/4277660322 learned_addr learned my addr 172.21.15.43:0/4277660322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1a00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84005210 0x7f5e9c1e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 -- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84005210 msgr2=0x7f5e9c1e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84005210 0x7f5e9c1e8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea2400640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9c144aa0 0x7f5e9c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 -- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9c144aa0 msgr2=0x7f5e9c144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9c144aa0 0x7f5e9c144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.818+0000 7f5ea1000640 1 -- 172.21.15.43:0/4277660322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c1f4d30 con 0x7f5e9c1e8110 2024-04-26T20:32:19.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5ea1000640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800d770 tx=0x7f5e8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88019070 con 0x7f5e9c1e8110 2024-04-26T20:32:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e880072d0 con 0x7f5e9c1e8110 2024-04-26T20:32:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880047e0 con 0x7f5e9c1e8110 2024-04-26T20:32:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c1e7440 con 0x7f5e9c1e8110 2024-04-26T20:32:19.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.819+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c1e7a00 con 0x7f5e9c1e8110 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.820+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e9c069350 con 0x7f5e9c1e8110 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.821+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e8801c070 con 0x7f5e9c1e8110 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.823+0000 7f5e99600640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 0x7f5e7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.823+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f5e8808b370 con 0x7f5e9c1e8110 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.823+0000 7f5ea1a00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 0x7f5e7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.824+0000 7f5ea1a00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 0x7f5e7c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5e90001630 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:19.827+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e9c072a80 con 0x7f5e9c1e8110 2024-04-26T20:32:20.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.032+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5e9c1f4f00 con 0x7f5e7c06c8f0 2024-04-26T20:32:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.033+0000 7f5e99600640 1 -- 172.21.15.43:0/4277660322 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5e9c1f4f00 con 0x7f5e7c06c8f0 2024-04-26T20:32:20.034 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 msgr2=0x7f5e7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 0x7f5e7c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5e90001630 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 msgr2=0x7f5e9c143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800d770 tx=0x7f5e8800dc30 comp rx=0 tx=0).stop 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 shutdown_connections 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e7c06c8f0 0x7f5e7c06edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9c144aa0 0x7f5e9c144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9c1e8110 0x7f5e9c143180 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 --2- 172.21.15.43:0/4277660322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e84005210 0x7f5e9c1e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 >> 172.21.15.43:0/4277660322 conn(0x7f5e9c068090 msgr2=0x7f5e9c0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.035+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 shutdown_connections 2024-04-26T20:32:20.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.036+0000 7f5ea2e00640 1 -- 172.21.15.43:0/4277660322 wait complete. 2024-04-26T20:32:20.057 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T20:32:20.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.279+0000 7f6b5ea00640 1 Processor -- start 2024-04-26T20:32:20.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.279+0000 7f6b5ea00640 1 -- start start 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5ea00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b58145450 0x7f6b5806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b58071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5ea00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b580738c0 con 0x7f6b58160090 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5ea00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6b58073e40 con 0x7f6b58145450 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6b58073b60 con 0x7f6b580b52d0 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5d600640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b58145450 0x7f6b5806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5cc00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b58160090 0x7f6b580b4bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59272/0 (socket says 172.21.15.43:59272) 2024-04-26T20:32:20.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.281+0000 7f6b5cc00640 1 -- 172.21.15.43:0/1608423071 learned_addr learned my addr 172.21.15.43:0/1608423071 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b5e000640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b58071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b57e00640 1 -- 172.21.15.43:0/1608423071 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826617143 0 0) 0x7f6b580738c0 con 0x7f6b58160090 2024-04-26T20:32:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b57e00640 1 -- 172.21.15.43:0/1608423071 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b40003520 con 0x7f6b58160090 2024-04-26T20:32:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b5d600640 1 -- 172.21.15.43:0/1608423071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 msgr2=0x7f6b58071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:20.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b5d600640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b58071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b5d600640 1 -- 172.21.15.43:0/1608423071 >> v1:172.21.15.43:6789/0 conn(0x7f6b58160090 legacy=0x7f6b580b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.282+0000 7f6b5d600640 1 -- 172.21.15.43:0/1608423071 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5806ebe0 con 0x7f6b58145450 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.283+0000 7f6b5d600640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b5806f6c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042747434cf650c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b57e00640 1 -- 172.21.15.43:0/1608423071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c037070 con 0x7f6b58145450 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b57e00640 1 -- 172.21.15.43:0/1608423071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c032440 con 0x7f6b58145450 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b57e00640 1 -- 172.21.15.43:0/1608423071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c032810 con 0x7f6b58145450 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 -- 172.21.15.43:0/1608423071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 msgr2=0x7f6b5806f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b5806f6c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02a180 comp rx=0 tx=0).stop 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 -- 172.21.15.43:0/1608423071 shutdown_connections 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b58071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/1608423071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b5806f6c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 -- 172.21.15.43:0/1608423071 >> 172.21.15.43:0/1608423071 conn(0x7f6b580b26b0 msgr2=0x7f6b580b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.284+0000 7f6b5ea00640 1 -- 172.21.15.43:0/1608423071 shutdown_connections 2024-04-26T20:32:20.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.285+0000 7f6b5ea00640 1 -- 172.21.15.43:0/1608423071 wait complete. 2024-04-26T20:32:20.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.286+0000 7f6b5ea00640 1 Processor -- start 2024-04-26T20:32:20.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.286+0000 7f6b5ea00640 1 -- start start 2024-04-26T20:32:20.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b580c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b58140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58160090 0x7f6b581f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6b580cd4e0 con 0x7f6b58145450 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58160540 con 0x7f6b58160090 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.287+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6b580cd660 con 0x7f6b580b52d0 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b58140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b58140d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34602/0 (socket says 172.21.15.43:34602) 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 -- 172.21.15.43:0/71456407 learned_addr learned my addr 172.21.15.43:0/71456407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 -- 172.21.15.43:0/71456407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 msgr2=0x7f6b580c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6b580b52d0 0x7f6b580c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 -- 172.21.15.43:0/71456407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58160090 msgr2=0x7f6b581f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6b58160090 0x7f6b581f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b58142890 con 0x7f6b58145450 2024-04-26T20:32:20.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b5cc00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b58140d50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c460 tx=0x7f6b4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.288+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48017070 con 0x7f6b58145450 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.289+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b48005ce0 con 0x7f6b58145450 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.289+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48007290 con 0x7f6b58145450 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.289+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b581417f0 con 0x7f6b58145450 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.289+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b58141db0 con 0x7f6b58145450 2024-04-26T20:32:20.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.289+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b580cdae0 con 0x7f6b58145450 2024-04-26T20:32:20.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.291+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b4801a070 con 0x7f6b58145450 2024-04-26T20:32:20.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.292+0000 7f6b55600640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.292+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f6b4808ad80 con 0x7f6b58145450 2024-04-26T20:32:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.292+0000 7f6b5d600640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.292+0000 7f6b55600640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6b3c072d90 0x7f6b3c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.293+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b3c0758a0 con 0x7f6b3c072d90 2024-04-26T20:32:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.293+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6b4808f030 con 0x7f6b58145450 2024-04-26T20:32:20.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.293+0000 7f6b5e000640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6b3c072d90 0x7f6b3c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.293+0000 7f6b5d600640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c034000 tx=0x7f6b4c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.293+0000 7f6b5e000640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6b3c072d90 0x7f6b3c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.294+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== osd.3 v2:172.21.15.43:6816/429853617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b3c0758a0 con 0x7f6b3c072d90 2024-04-26T20:32:20.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.309+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 --> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b58142590 con 0x7f6b3c072d90 2024-04-26T20:32:20.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.309+0000 7f6b55600640 1 -- 172.21.15.43:0/71456407 <== osd.3 v2:172.21.15.43:6816/429853617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b50009a30 con 0x7f6b3c072d90 2024-04-26T20:32:20.310 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:20.310 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:20.310 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6b3c072d90 msgr2=0x7f6b3c0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6816/429853617,v1:172.21.15.43:6817/429853617] conn(0x7f6b3c072d90 0x7f6b3c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b3c06c8f0 msgr2=0x7f6b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c034000 tx=0x7f6b4c02c000 comp rx=0 tx=0).stop 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 msgr2=0x7f6b58140d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 --2- 172.21.15.43:0/71456407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58145450 0x7f6b58140d50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800c460 tx=0x7f6b4800c920 comp rx=0 tx=0).stop 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5e000640 1 -- 172.21.15.43:0/71456407 reap_dead start 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 shutdown_connections 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 >> 172.21.15.43:0/71456407 conn(0x7f6b580b26b0 msgr2=0x7f6b580cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 shutdown_connections 2024-04-26T20:32:20.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.310+0000 7f6b5ea00640 1 -- 172.21.15.43:0/71456407 wait complete. 2024-04-26T20:32:20.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T20:32:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.526+0000 7fde40800640 1 Processor -- start 2024-04-26T20:32:20.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.527+0000 7fde40800640 1 -- start start 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde3815cab0 0x7fde380afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde380b0020 0x7fde380d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde380ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fde380d14f0 con 0x7fde380b0020 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fde380d0900 con 0x7fde3815cab0 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.528+0000 7fde40800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fde381635a0 con 0x7fde380d1080 2024-04-26T20:32:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde380b0020 0x7fde380d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde380b0020 0x7fde380d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34610/0 (socket says 172.21.15.43:34610) 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 -- 172.21.15.43:0/4078011858 learned_addr learned my addr 172.21.15.43:0/4078011858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 -- 172.21.15.43:0/4078011858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 msgr2=0x7fde380ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 --2- 172.21.15.43:0/4078011858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde380ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 -- 172.21.15.43:0/4078011858 >> v2:172.21.15.164:3300/0 conn(0x7fde3815cab0 msgr2=0x7fde380afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 --2- 172.21.15.43:0/4078011858 >> v2:172.21.15.164:3300/0 conn(0x7fde3815cab0 0x7fde380afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.529+0000 7fde3d400640 1 -- 172.21.15.43:0/4078011858 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380b0650 con 0x7fde380b0020 2024-04-26T20:32:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.530+0000 7fde3d400640 1 --2- 172.21.15.43:0/4078011858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde380d0b40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde3806d120 tx=0x7fde2802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f0ebc26c4bc58d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde3ca00640 1 -- 172.21.15.43:0/4078011858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28037070 con 0x7fde380b0020 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde3ca00640 1 -- 172.21.15.43:0/4078011858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde28032440 con 0x7fde380b0020 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde3ca00640 1 -- 172.21.15.43:0/4078011858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28032810 con 0x7fde380b0020 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde40800640 1 -- 172.21.15.43:0/4078011858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 msgr2=0x7fde380d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde40800640 1 --2- 172.21.15.43:0/4078011858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde380d0b40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fde3806d120 tx=0x7fde2802a1d0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.531+0000 7fde40800640 1 -- 172.21.15.43:0/4078011858 shutdown_connections 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 --2- 172.21.15.43:0/4078011858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde380ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 --2- 172.21.15.43:0/4078011858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde380d0b40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 --2- 172.21.15.43:0/4078011858 >> v2:172.21.15.164:3300/0 conn(0x7fde3815cab0 0x7fde380afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 -- 172.21.15.43:0/4078011858 >> 172.21.15.43:0/4078011858 conn(0x7fde380b2900 msgr2=0x7fde380b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 -- 172.21.15.43:0/4078011858 shutdown_connections 2024-04-26T20:32:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.532+0000 7fde40800640 1 -- 172.21.15.43:0/4078011858 wait complete. 2024-04-26T20:32:20.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.534+0000 7fde40800640 1 Processor -- start 2024-04-26T20:32:20.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.534+0000 7fde40800640 1 -- start start 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde38159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3815cab0 0x7fde38144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde38162f50 con 0x7fde380b0020 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde38162c70 con 0x7fde3815cab0 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde40800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fde38163230 con 0x7fde380d1080 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.535+0000 7fde3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34620/0 (socket says 172.21.15.43:34620) 2024-04-26T20:32:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 -- 172.21.15.43:0/1315205153 learned_addr learned my addr 172.21.15.43:0/1315205153 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 -- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 msgr2=0x7fde38159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde38159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 -- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3815cab0 msgr2=0x7fde38144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3815cab0 0x7fde38144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 -- 172.21.15.43:0/1315205153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3815fd60 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde3de00640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00b440 tx=0x7fde2c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.536+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c015020 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.537+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c011890 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.537+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde38160070 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.537+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011c80 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.537+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381608b0 con 0x7fde380b0020 2024-04-26T20:32:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.538+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3806dc30 con 0x7fde380b0020 2024-04-26T20:32:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.539+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde2c018020 con 0x7fde380b0020 2024-04-26T20:32:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.540+0000 7fde32000640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 0x7fde1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.540+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fde2c08bb70 con 0x7fde380b0020 2024-04-26T20:32:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.540+0000 7fde3d400640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 0x7fde1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.541+0000 7fde3d400640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 0x7fde1c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fde28034000 tx=0x7fde2802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.544+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde3806dc30 con 0x7fde380b0020 2024-04-26T20:32:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.743+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fde38069350 con 0x7fde1c06c8f0 2024-04-26T20:32:20.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.744+0000 7fde32000640 1 -- 172.21.15.43:0/1315205153 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fde38069350 con 0x7fde1c06c8f0 2024-04-26T20:32:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 msgr2=0x7fde1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 0x7fde1c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fde28034000 tx=0x7fde2802c000 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 msgr2=0x7fde3815f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00b440 tx=0x7fde2c00b900 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 shutdown_connections 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fde1c06c8f0 0x7fde1c06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde380d1080 0x7fde38159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3815cab0 0x7fde38144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 --2- 172.21.15.43:0/1315205153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde380b0020 0x7fde3815f430 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 >> 172.21.15.43:0/1315205153 conn(0x7fde380b2900 msgr2=0x7fde3806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 shutdown_connections 2024-04-26T20:32:20.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.747+0000 7fde40800640 1 -- 172.21.15.43:0/1315205153 wait complete. 2024-04-26T20:32:20.767 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T20:32:20.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:20.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.976+0000 7f28ebe00640 1 Processor -- start 2024-04-26T20:32:20.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.977+0000 7f28ebe00640 1 -- start start 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.978+0000 7f28ebe00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28ec15f0a0 0x7f28ec0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.978+0000 7f28ebe00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ec0b4310 con 0x7f28ec15f480 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.978+0000 7f28ebe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28ec0d0f70 con 0x7f28ec0d1dd0 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.978+0000 7f28ebe00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f28ec0d1250 con 0x7f28ec15f0a0 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28eaa00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28ec15f0a0 0x7f28ec0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28ea000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f28ec15f480 0x7f28ec0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59274/0 (socket says 172.21.15.43:59274) 2024-04-26T20:32:20.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28ea000640 1 -- 172.21.15.43:0/2259486474 learned_addr learned my addr 172.21.15.43:0/2259486474 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:20.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28eaa00640 1 -- 172.21.15.43:0/2259486474 >> v1:172.21.15.164:6789/0 conn(0x7f28ec0d1dd0 legacy=0x7f28ec0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:20.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28eaa00640 1 -- 172.21.15.43:0/2259486474 >> v1:172.21.15.43:6789/0 conn(0x7f28ec15f480 legacy=0x7f28ec0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.979+0000 7f28eaa00640 1 -- 172.21.15.43:0/2259486474 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0b45d0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.980+0000 7f28eaa00640 1 --2- 172.21.15.43:0/2259486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0d2b70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0097a0 tx=0x7f28dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af2bb97602e2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28e9600640 1 -- 172.21.15.43:0/2259486474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc037070 con 0x7f28ec15f0a0 2024-04-26T20:32:20.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28e9600640 1 -- 172.21.15.43:0/2259486474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc032440 con 0x7f28ec15f0a0 2024-04-26T20:32:20.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28e9600640 1 -- 172.21.15.43:0/2259486474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc032810 con 0x7f28ec15f0a0 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28ebe00640 1 -- 172.21.15.43:0/2259486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 msgr2=0x7f28ec0d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28ebe00640 1 --2- 172.21.15.43:0/2259486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0d2b70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0097a0 tx=0x7f28dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28ebe00640 1 -- 172.21.15.43:0/2259486474 shutdown_connections 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28ebe00640 1 --2- 172.21.15.43:0/2259486474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0d2b70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.981+0000 7f28ebe00640 1 -- 172.21.15.43:0/2259486474 >> 172.21.15.43:0/2259486474 conn(0x7f28ec09dff0 msgr2=0x7f28ec0d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.982+0000 7f28ebe00640 1 -- 172.21.15.43:0/2259486474 shutdown_connections 2024-04-26T20:32:20.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.982+0000 7f28ebe00640 1 -- 172.21.15.43:0/2259486474 wait complete. 2024-04-26T20:32:20.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.983+0000 7f28ebe00640 1 Processor -- start 2024-04-26T20:32:20.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.984+0000 7f28ebe00640 1 -- start start 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.984+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ec0d1dd0 0x7f28ec0ba210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.984+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ebe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec15f480 0x7f28ec159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ebe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0d5af0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ebe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec0d30b0 con 0x7f28ec15f480 2024-04-26T20:32:20.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ebe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f28ec0d0b90 con 0x7f28ec0d1dd0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34636/0 (socket says 172.21.15.43:34636) 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 -- 172.21.15.43:0/246586328 learned_addr learned my addr 172.21.15.43:0/246586328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28eb400640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec15f480 0x7f28ec159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 -- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ec0d1dd0 msgr2=0x7f28ec0ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ec0d1dd0 0x7f28ec0ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 -- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec15f480 msgr2=0x7f28ec159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.985+0000 7f28ea000640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ec15f480 0x7f28ec159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28ea000640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec0b9770 con 0x7f28ec15f0a0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28ea000640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0b5d70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c3a0 tx=0x7f28d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4017070 con 0x7f28ec15f0a0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d4005ce0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec0ce910 con 0x7f28ec15f0a0 2024-04-26T20:32:20.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40072f0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.986+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec0b9cf0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.987+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28ec0d4a60 con 0x7f28ec15f0a0 2024-04-26T20:32:20.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.988+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28d401a070 con 0x7f28ec15f0a0 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.989+0000 7f28e2a00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f28c806c8f0 0x7f28c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.989+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f28d408ae20 con 0x7f28ec15f0a0 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.990+0000 7f28eaa00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f28c806c8f0 0x7f28c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.990+0000 7f28e2a00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f28c8072d90 0x7f28c80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.990+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28c80758a0 con 0x7f28c8072d90 2024-04-26T20:32:20.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.990+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f28d408d3e0 con 0x7f28ec15f0a0 2024-04-26T20:32:20.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.990+0000 7f28eaa00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f28c806c8f0 0x7f28c806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034000 tx=0x7f28dc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.991+0000 7f28eb400640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f28c8072d90 0x7f28c80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:20.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.992+0000 7f28eb400640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f28c8072d90 0x7f28c80751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:20.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:20.994+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== osd.4 v2:172.21.15.164:6808/2254882506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28c80758a0 con 0x7f28c8072d90 2024-04-26T20:32:21.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.009+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 --> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f28ec08dcd0 con 0x7f28c8072d90 2024-04-26T20:32:21.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28e2a00640 1 -- 172.21.15.43:0/246586328 <== osd.4 v2:172.21.15.164:6808/2254882506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f28ec08dcd0 con 0x7f28c8072d90 2024-04-26T20:32:21.010 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:21.010 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:21.011 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f28c8072d90 msgr2=0x7f28c80751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.164:6808/2254882506,v1:172.21.15.164:6809/2254882506] conn(0x7f28c8072d90 0x7f28c80751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f28c806c8f0 msgr2=0x7f28c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f28c806c8f0 0x7f28c806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f28dc034000 tx=0x7f28dc02c000 comp rx=0 tx=0).stop 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 msgr2=0x7f28ec0b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 --2- 172.21.15.43:0/246586328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28ec15f0a0 0x7f28ec0b5d70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f28d400c3a0 tx=0x7f28d400c860 comp rx=0 tx=0).stop 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28eb400640 1 -- 172.21.15.43:0/246586328 reap_dead start 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 shutdown_connections 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.010+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 >> 172.21.15.43:0/246586328 conn(0x7f28ec09dff0 msgr2=0x7f28ec15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.011+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 shutdown_connections 2024-04-26T20:32:21.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.011+0000 7f28ebe00640 1 -- 172.21.15.43:0/246586328 wait complete. 2024-04-26T20:32:21.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T20:32:21.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.225+0000 7f8d55400640 1 Processor -- start 2024-04-26T20:32:21.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.226+0000 7f8d55400640 1 -- start start 2024-04-26T20:32:21.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.227+0000 7f8d55400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d501628e0 0x7f8d50162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d55400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d500af720 0x7f8d500afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d55400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d50145710 con 0x7f8d50145330 2024-04-26T20:32:21.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d55400640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8d50163280 con 0x7f8d500af720 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d55400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8d50073dc0 con 0x7f8d501628e0 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d4fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8d50145330 0x7f8d500b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59284/0 (socket says 172.21.15.43:59284) 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d4fe00640 1 -- 172.21.15.43:0/1830457681 learned_addr learned my addr 172.21.15.43:0/1830457681 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d54a00640 1 --2- 172.21.15.43:0/1830457681 >> v2:172.21.15.43:3300/0 conn(0x7f8d500af720 0x7f8d500afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d4f400640 1 -- 172.21.15.43:0/1830457681 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556061895 0 0) 0x7f8d50145710 con 0x7f8d50145330 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d47400640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d501628e0 0x7f8d50162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d4f400640 1 -- 172.21.15.43:0/1830457681 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d34003520 con 0x7f8d50145330 2024-04-26T20:32:21.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.228+0000 7f8d54a00640 1 -- 172.21.15.43:0/1830457681 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d501628e0 msgr2=0x7f8d50162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.229+0000 7f8d54a00640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d501628e0 0x7f8d50162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.229+0000 7f8d54a00640 1 -- 172.21.15.43:0/1830457681 >> v1:172.21.15.43:6789/0 conn(0x7f8d50145330 legacy=0x7f8d500b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.229+0000 7f8d54a00640 1 -- 172.21.15.43:0/1830457681 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d500ae960 con 0x7f8d500af720 2024-04-26T20:32:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d54a00640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d500afbb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097a0 tx=0x7f8d4802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fcaf285eaebf93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d4f400640 1 -- 172.21.15.43:0/1830457681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48037070 con 0x7f8d500af720 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d4f400640 1 -- 172.21.15.43:0/1830457681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d48032440 con 0x7f8d500af720 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d4f400640 1 -- 172.21.15.43:0/1830457681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48032810 con 0x7f8d500af720 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d55400640 1 -- 172.21.15.43:0/1830457681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 msgr2=0x7f8d500afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.230+0000 7f8d55400640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d500afbb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097a0 tx=0x7f8d4802a180 comp rx=0 tx=0).stop 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 -- 172.21.15.43:0/1830457681 shutdown_connections 2024-04-26T20:32:21.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d500afbb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 --2- 172.21.15.43:0/1830457681 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d501628e0 0x7f8d50162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 -- 172.21.15.43:0/1830457681 >> 172.21.15.43:0/1830457681 conn(0x7f8d50068090 msgr2=0x7f8d50068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 -- 172.21.15.43:0/1830457681 shutdown_connections 2024-04-26T20:32:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.231+0000 7f8d55400640 1 -- 172.21.15.43:0/1830457681 wait complete. 2024-04-26T20:32:21.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.232+0000 7f8d55400640 1 Processor -- start 2024-04-26T20:32:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.232+0000 7f8d55400640 1 -- start start 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 0x7f8d501405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d501628e0 0x7f8d500c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d500af200 con 0x7f8d500af720 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d500aef20 con 0x7f8d501628e0 2024-04-26T20:32:21.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.233+0000 7f8d55400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8d500b00f0 con 0x7f8d50145330 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34658/0 (socket says 172.21.15.43:34658) 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 -- 172.21.15.43:0/510894619 learned_addr learned my addr 172.21.15.43:0/510894619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d47400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 0x7f8d501405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 -- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 msgr2=0x7f8d501405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 0x7f8d501405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 -- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d501628e0 msgr2=0x7f8d500c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d501628e0 0x7f8d500c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 -- 172.21.15.43:0/510894619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5012eed0 con 0x7f8d500af720 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d47400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 0x7f8d501405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.234+0000 7f8d4fe00640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000cb40 tx=0x7f8d4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.235+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40017070 con 0x7f8d500af720 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.235+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4000aca0 con 0x7f8d500af720 2024-04-26T20:32:21.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.235+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40012620 con 0x7f8d500af720 2024-04-26T20:32:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.235+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d50141070 con 0x7f8d500af720 2024-04-26T20:32:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.235+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d50134ff0 con 0x7f8d500af720 2024-04-26T20:32:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.236+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d50069350 con 0x7f8d500af720 2024-04-26T20:32:21.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.237+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d4001a070 con 0x7f8d500af720 2024-04-26T20:32:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.239+0000 7f8d4cc00640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 0x7f8d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.239+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8d4008b930 con 0x7f8d500af720 2024-04-26T20:32:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.239+0000 7f8d47400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 0x7f8d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.240+0000 7f8d47400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 0x7f8d2c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.243+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d40059e30 con 0x7f8d500af720 2024-04-26T20:32:21.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.446+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8d500d0260 con 0x7f8d2c06c8f0 2024-04-26T20:32:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.447+0000 7f8d4cc00640 1 -- 172.21.15.43:0/510894619 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8d500d0260 con 0x7f8d2c06c8f0 2024-04-26T20:32:21.447 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 msgr2=0x7f8d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 0x7f8d2c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006fd0 tx=0x7f8d38008040 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 msgr2=0x7f8d501f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000cb40 tx=0x7f8d4000a060 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 shutdown_connections 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f8d2c06c8f0 0x7f8d2c06edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d50145330 0x7f8d501405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d501628e0 0x7f8d500c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 --2- 172.21.15.43:0/510894619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d500af720 0x7f8d501f4150 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 >> 172.21.15.43:0/510894619 conn(0x7f8d50068090 msgr2=0x7f8d50072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 shutdown_connections 2024-04-26T20:32:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.452+0000 7f8d55400640 1 -- 172.21.15.43:0/510894619 wait complete. 2024-04-26T20:32:21.472 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T20:32:21.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:21.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.685+0000 7f5836600640 1 Processor -- start 2024-04-26T20:32:21.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.685+0000 7f5836600640 1 -- start start 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.686+0000 7f5836600640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f58300ce0f0 0x7f58300b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f5836600640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f583015d260 con 0x7f58300cdc80 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f5836600640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f583015d540 con 0x7f58300b00a0 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f5836600640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f583015d820 con 0x7f58300ce0f0 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f582fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f58300cdc80 0x7f58300b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59300/0 (socket says 172.21.15.43:59300) 2024-04-26T20:32:21.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f582fe00640 1 -- 172.21.15.43:0/230288355 learned_addr learned my addr 172.21.15.43:0/230288355 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:21.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f5835c00640 1 --2- 172.21.15.43:0/230288355 >> v2:172.21.15.164:3300/0 conn(0x7f58300ce0f0 0x7f58300b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951762012 0 0) 0x7f583015d260 con 0x7f58300cdc80 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.687+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5818003520 con 0x7f58300cdc80 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2156487274 0 0) 0x7f5818003520 con 0x7f58300cdc80 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 >> v2:172.21.15.164:3300/0 conn(0x7f58300ce0f0 msgr2=0x7f58300b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 --2- 172.21.15.43:0/230288355 >> v2:172.21.15.164:3300/0 conn(0x7f58300ce0f0 0x7f58300b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 >> v1:172.21.15.164:6789/0 conn(0x7f58300b00a0 legacy=0x7f58300cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583015db00 con 0x7f58300cdc80 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f581c001670 con 0x7f58300cdc80 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 >> v1:172.21.15.43:6789/0 conn(0x7f58300cdc80 legacy=0x7f58300b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58180039b0 0x7f5818003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5818004800 0x7f5818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 0x7f581800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f583015d820 con 0x7f58180051c0 2024-04-26T20:32:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f581800c060 con 0x7f58180039b0 2024-04-26T20:32:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f581800c260 con 0x7f5818004800 2024-04-26T20:32:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.688+0000 7f5835c00640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 0x7f581800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835200640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58180039b0 0x7f5818003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835c00640 1 -- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5818004800 msgr2=0x7f5818004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835c00640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5818004800 0x7f5818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835c00640 1 -- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58180039b0 msgr2=0x7f5818003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835c00640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58180039b0 0x7f5818003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835c00640 1 -- 172.21.15.43:0/230288355 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5818003520 con 0x7f58180051c0 2024-04-26T20:32:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.689+0000 7f5835200640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58180039b0 0x7f5818003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.690+0000 7f5835c00640 1 -- 172.21.15.43:0/230288355 reap_dead start 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.690+0000 7f5835c00640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 0x7f581800b9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5828009fc0 tx=0x7f5818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6065da37fc51760 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.690+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828035020 con 0x7f58180051c0 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.690+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5828003b90 con 0x7f58180051c0 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.690+0000 7f582f400640 1 -- 172.21.15.43:0/230288355 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828009c10 con 0x7f58180051c0 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 -- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 msgr2=0x7f581800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 0x7f581800b9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5828009fc0 tx=0x7f5818005830 comp rx=0 tx=0).stop 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 -- 172.21.15.43:0/230288355 shutdown_connections 2024-04-26T20:32:21.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 --2- 172.21.15.43:0/230288355 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58180051c0 0x7f581800b9c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 -- 172.21.15.43:0/230288355 >> 172.21.15.43:0/230288355 conn(0x7f58300b26b0 msgr2=0x7f58300b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 -- 172.21.15.43:0/230288355 shutdown_connections 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.691+0000 7f5836600640 1 -- 172.21.15.43:0/230288355 wait complete. 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.693+0000 7f5836600640 1 Processor -- start 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.693+0000 7f5836600640 1 -- start start 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58180051c0 0x7f5830139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 0x7f583015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5830144db0 0x7f5830145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f583006f7f0 con 0x7f58301f1990 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f583006f670 con 0x7f5830144db0 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5836600640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f58300cfe70 con 0x7f58180051c0 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 0x7f583015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 0x7f583015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34676/0 (socket says 172.21.15.43:34676) 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 -- 172.21.15.43:0/2360223430 learned_addr learned my addr 172.21.15.43:0/2360223430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f5835c00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5830144db0 0x7f5830145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 -- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58180051c0 msgr2=0x7f5830139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58180051c0 0x7f5830139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 -- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5830144db0 msgr2=0x7f5830145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.694+0000 7f582fe00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5830144db0 0x7f5830145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f582fe00640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5830160d90 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f582fe00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 0x7f583015eb00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c700 tx=0x7f581c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c018070 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f581c0074e0 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301607d0 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c004a20 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.695+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583015e300 con 0x7f58301f1990 2024-04-26T20:32:21.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.696+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58301f1e10 con 0x7f58301f1990 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.698+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f581c01b070 con 0x7f58301f1990 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.698+0000 7f582cc00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f581006c8f0 0x7f581006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f581c08aeb0 con 0x7f58301f1990 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f582cc00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f5810072d90 0x7f58100751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f5835200640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f581006c8f0 0x7f581006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f581c00ae50 con 0x7f5810072d90 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f581c090480 con 0x7f58301f1990 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f5835c00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f5810072d90 0x7f58100751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.699+0000 7f5835200640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f581006c8f0 0x7f581006edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f582400c810 tx=0x7f58240079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.700+0000 7f5835c00640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f5810072d90 0x7f58100751d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.703+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== osd.5 v2:172.21.15.164:6800/2479038766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f581c00ae50 con 0x7f5810072d90 2024-04-26T20:32:21.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.717+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 --> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5830069350 con 0x7f5810072d90 2024-04-26T20:32:21.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.718+0000 7f582cc00640 1 -- 172.21.15.43:0/2360223430 <== osd.5 v2:172.21.15.164:6800/2479038766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5830069350 con 0x7f5810072d90 2024-04-26T20:32:21.719 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:21.719 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:21.719 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.718+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f5810072d90 msgr2=0x7f58100751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.718+0000 7f5836600640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.164:6800/2479038766,v1:172.21.15.164:6801/2479038766] conn(0x7f5810072d90 0x7f58100751d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f581006c8f0 msgr2=0x7f581006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f581006c8f0 0x7f581006edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f582400c810 tx=0x7f58240079c0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 msgr2=0x7f583015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 --2- 172.21.15.43:0/2360223430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58301f1990 0x7f583015eb00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c700 tx=0x7f581c00cbc0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5835c00640 1 -- 172.21.15.43:0/2360223430 reap_dead start 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 shutdown_connections 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 >> 172.21.15.43:0/2360223430 conn(0x7f58300b26b0 msgr2=0x7f5830142c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 shutdown_connections 2024-04-26T20:32:21.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.719+0000 7f5836600640 1 -- 172.21.15.43:0/2360223430 wait complete. 2024-04-26T20:32:21.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T20:32:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.939+0000 7f085b200640 1 Processor -- start 2024-04-26T20:32:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.939+0000 7f085b200640 1 -- start start 2024-04-26T20:32:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f085b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f085c0d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f085b200640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0d0480 con 0x7f085c0b5b80 2024-04-26T20:32:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f085b200640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0d0720 con 0x7f085c0d1620 2024-04-26T20:32:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f085b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f085c0d5be0 con 0x7f085c0d5750 2024-04-26T20:32:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f0859400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f085c0b5b80 0x7f085c0d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59316/0 (socket says 172.21.15.43:59316) 2024-04-26T20:32:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f0859400640 1 -- 172.21.15.43:0/3062407557 learned_addr learned my addr 172.21.15.43:0/3062407557 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.941+0000 7f085a800640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f085c0d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085296210 0 0) 0x7f085c0d0480 con 0x7f085c0b5b80 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0840003520 con 0x7f085c0b5b80 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3715713680 0 0) 0x7f0840003520 con 0x7f085c0b5b80 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 msgr2=0x7f085c0d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f085c0d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 >> v1:172.21.15.164:6789/0 conn(0x7f085c0d1620 legacy=0x7f085c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c0d0a00 con 0x7f085c0b5b80 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7f0848001670 con 0x7f085c0b5b80 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 >> v1:172.21.15.43:6789/0 conn(0x7f085c0b5b80 legacy=0x7f085c0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f08400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840003fb0 0x7f0840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0840004d40 0x7f08400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0d5be0 con 0x7f0840004d40 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f084000bbb0 con 0x7f0840003fb0 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.942+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f084000bdb0 con 0x7f085c0d5750 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0840004d40 0x7f08400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f0859e00640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f08400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f0859400640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840003fb0 0x7f0840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 -- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 msgr2=0x7f08400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c0d5750 0x7f08400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 -- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840003fb0 msgr2=0x7f0840004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840003fb0 0x7f0840004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.943+0000 7f085a800640 1 -- 172.21.15.43:0/3062407557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0840003520 con 0x7f0840004d40 2024-04-26T20:32:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.944+0000 7f085a800640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0840004d40 0x7f08400051c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f084c009ac0 tx=0x7f0840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86c7da7b3abc2de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c036070 con 0x7f0840004d40 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c031400 con 0x7f0840004d40 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f0858a00640 1 -- 172.21.15.43:0/3062407557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c009690 con 0x7f0840004d40 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 -- 172.21.15.43:0/3062407557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0840004d40 msgr2=0x7f08400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 --2- 172.21.15.43:0/3062407557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0840004d40 0x7f08400051c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f084c009ac0 tx=0x7f0840005830 comp rx=0 tx=0).stop 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085a800640 1 -- 172.21.15.43:0/3062407557 reap_dead start 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 -- 172.21.15.43:0/3062407557 shutdown_connections 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 -- 172.21.15.43:0/3062407557 >> 172.21.15.43:0/3062407557 conn(0x7f085c0d4c30 msgr2=0x7f085c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 -- 172.21.15.43:0/3062407557 shutdown_connections 2024-04-26T20:32:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.945+0000 7f085b200640 1 -- 172.21.15.43:0/3062407557 wait complete. 2024-04-26T20:32:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.947+0000 7f085b200640 1 Processor -- start 2024-04-26T20:32:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.947+0000 7f085b200640 1 -- start start 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840004d40 0x7f085c1f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c1f8920 0x7f085c09e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0d2f10 con 0x7f085c0ce3a0 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0d0ce0 con 0x7f0840004d40 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.948+0000 7f085b200640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f085c0d3090 con 0x7f085c1f8920 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34692/0 (socket says 172.21.15.43:34692) 2024-04-26T20:32:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 -- 172.21.15.43:0/2766557963 learned_addr learned my addr 172.21.15.43:0/2766557963 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f0859400640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c1f8920 0x7f085c09e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f0859e00640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840004d40 0x7f085c1f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 -- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c1f8920 msgr2=0x7f085c09e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c1f8920 0x7f085c09e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 -- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840004d40 msgr2=0x7f085c1f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840004d40 0x7f085c1f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.949+0000 7f085a800640 1 -- 172.21.15.43:0/2766557963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c1f50f0 con 0x7f085c0ce3a0 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f085a800640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f084c009be0 tx=0x7f084c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c036040 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c041070 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c009690 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c1f53d0 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.950+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c0b9840 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.951+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c0b9a40 con 0x7f085c0ce3a0 2024-04-26T20:32:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.952+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084c039070 con 0x7f085c0ce3a0 2024-04-26T20:32:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.953+0000 7f0852000640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.953+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f084c0ab240 con 0x7f085c0ce3a0 2024-04-26T20:32:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.954+0000 7f0859e00640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.954+0000 7f0859e00640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 0x7f083c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f085c0b1ba0 tx=0x7f0854009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:21.958+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f085c0b9a40 con 0x7f085c0ce3a0 2024-04-26T20:32:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.163+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f085c1f58f0 con 0x7f083c06c8f0 2024-04-26T20:32:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.163+0000 7f0852000640 1 -- 172.21.15.43:0/2766557963 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f08540057e0 con 0x7f083c06c8f0 2024-04-26T20:32:22.164 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 msgr2=0x7f083c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 0x7f083c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f085c0b1ba0 tx=0x7f0854009040 comp rx=0 tx=0).stop 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 msgr2=0x7f085c152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f084c009be0 tx=0x7f084c004730 comp rx=0 tx=0).stop 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 shutdown_connections 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f083c06c8f0 0x7f083c06edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f085c1f8920 0x7f085c09e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f085c0ce3a0 0x7f085c152f40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 --2- 172.21.15.43:0/2766557963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0840004d40 0x7f085c1f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.166+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 >> 172.21.15.43:0/2766557963 conn(0x7f085c0d4c30 msgr2=0x7f085c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:22.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.167+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 shutdown_connections 2024-04-26T20:32:22.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.167+0000 7f085b200640 1 -- 172.21.15.43:0/2766557963 wait complete. 2024-04-26T20:32:22.186 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T20:32:22.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.407+0000 7fbb70e00640 1 Processor -- start 2024-04-26T20:32:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.407+0000 7fbb70e00640 1 -- start start 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb70e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6c146800 0x7fbb6c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb70e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb6c071e60 0x7fbb6c072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb70e00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6c06df30 con 0x7fbb6c06fec0 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb70e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6c146be0 con 0x7fbb6c146800 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb70e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6c06e860 con 0x7fbb6c071e60 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb69e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6c146800 0x7fbb6c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb69e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb6c146800 0x7fbb6c0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34708/0 (socket says 172.21.15.43:34708) 2024-04-26T20:32:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.409+0000 7fbb69e00640 1 -- 172.21.15.43:0/250620086 learned_addr learned my addr 172.21.15.43:0/250620086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.410+0000 7fbb69e00640 1 -- 172.21.15.43:0/250620086 >> v2:172.21.15.164:3300/0 conn(0x7fbb6c071e60 msgr2=0x7fbb6c072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.410+0000 7fbb69e00640 1 --2- 172.21.15.43:0/250620086 >> v2:172.21.15.164:3300/0 conn(0x7fbb6c071e60 0x7fbb6c072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.410+0000 7fbb69e00640 1 -- 172.21.15.43:0/250620086 >> v1:172.21.15.164:6789/0 conn(0x7fbb6c06fec0 legacy=0x7fbb6c06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.410+0000 7fbb69e00640 1 -- 172.21.15.43:0/250620086 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c06e210 con 0x7fbb6c146800 2024-04-26T20:32:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.411+0000 7fbb69e00640 1 --2- 172.21.15.43:0/250620086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c146800 0x7fbb6c0cfcd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c157a90 tx=0x7fbb5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bf16056f849ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.411+0000 7fbb68a00640 1 -- 172.21.15.43:0/250620086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c037070 con 0x7fbb6c146800 2024-04-26T20:32:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.411+0000 7fbb68a00640 1 -- 172.21.15.43:0/250620086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c032440 con 0x7fbb6c146800 2024-04-26T20:32:22.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.411+0000 7fbb68a00640 1 -- 172.21.15.43:0/250620086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c032810 con 0x7fbb6c146800 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 -- 172.21.15.43:0/250620086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c146800 msgr2=0x7fbb6c0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 --2- 172.21.15.43:0/250620086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c146800 0x7fbb6c0cfcd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c157a90 tx=0x7fbb5c02a230 comp rx=0 tx=0).stop 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 -- 172.21.15.43:0/250620086 shutdown_connections 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 --2- 172.21.15.43:0/250620086 >> v2:172.21.15.164:3300/0 conn(0x7fbb6c071e60 0x7fbb6c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 --2- 172.21.15.43:0/250620086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c146800 0x7fbb6c0cfcd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 -- 172.21.15.43:0/250620086 >> 172.21.15.43:0/250620086 conn(0x7fbb6c0b2790 msgr2=0x7fbb6c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 -- 172.21.15.43:0/250620086 shutdown_connections 2024-04-26T20:32:22.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.412+0000 7fbb70e00640 1 -- 172.21.15.43:0/250620086 wait complete. 2024-04-26T20:32:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.414+0000 7fbb70e00640 1 Processor -- start 2024-04-26T20:32:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.415+0000 7fbb70e00640 1 -- start start 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb6c06fec0 0x7fbb6c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 0x7fbb6c1f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb6c146800 0x7fbb6c1f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c0d0520 con 0x7fbb6c071e60 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c0d0210 con 0x7fbb6c06fec0 2024-04-26T20:32:22.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb70e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6c06f400 con 0x7fbb6c146800 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb69400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 0x7fbb6c1f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb69400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 0x7fbb6c1f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34712/0 (socket says 172.21.15.43:34712) 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb69400640 1 -- 172.21.15.43:0/3358151121 learned_addr learned my addr 172.21.15.43:0/3358151121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.416+0000 7fbb69e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb6c06fec0 0x7fbb6c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb6a800640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb6c146800 0x7fbb6c1f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 -- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb6c146800 msgr2=0x7fbb6c1f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb6c146800 0x7fbb6c1f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 -- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb6c06fec0 msgr2=0x7fbb6c1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb6c06fec0 0x7fbb6c1566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c1f94f0 con 0x7fbb6c071e60 2024-04-26T20:32:22.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb69400640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 0x7fbb6c1f5f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c410 tx=0x7fbb5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54017070 con 0x7fbb6c071e60 2024-04-26T20:32:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb540035f0 con 0x7fbb6c071e60 2024-04-26T20:32:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c1f97d0 con 0x7fbb6c071e60 2024-04-26T20:32:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.417+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540039e0 con 0x7fbb6c071e60 2024-04-26T20:32:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.418+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c1f5a10 con 0x7fbb6c071e60 2024-04-26T20:32:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.419+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb6c069350 con 0x7fbb6c071e60 2024-04-26T20:32:22.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.420+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb5401a070 con 0x7fbb6c071e60 2024-04-26T20:32:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.421+0000 7fbb62000640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbb4806c8f0 0x7fbb4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.421+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fbb5408aea0 con 0x7fbb6c071e60 2024-04-26T20:32:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.421+0000 7fbb62000640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7fbb48072d90 0x7fbb480751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.421+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb480758a0 con 0x7fbb48072d90 2024-04-26T20:32:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.421+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbb5408d460 con 0x7fbb6c071e60 2024-04-26T20:32:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.422+0000 7fbb69e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbb4806c8f0 0x7fbb4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.422+0000 7fbb6a800640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7fbb48072d90 0x7fbb480751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.423+0000 7fbb69e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbb4806c8f0 0x7fbb4806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c034000 tx=0x7fbb5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.424+0000 7fbb6a800640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7fbb48072d90 0x7fbb480751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.426+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== osd.6 v2:172.21.15.164:6816/3085699515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb480758a0 con 0x7fbb48072d90 2024-04-26T20:32:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.441+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 --> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbb6c073b10 con 0x7fbb48072d90 2024-04-26T20:32:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.442+0000 7fbb62000640 1 -- 172.21.15.43:0/3358151121 <== osd.6 v2:172.21.15.164:6816/3085699515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbb6c073b10 con 0x7fbb48072d90 2024-04-26T20:32:22.442 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:22.443 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:22.443 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7fbb48072d90 msgr2=0x7fbb480751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.164:6816/3085699515,v1:172.21.15.164:6817/3085699515] conn(0x7fbb48072d90 0x7fbb480751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbb4806c8f0 msgr2=0x7fbb4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fbb4806c8f0 0x7fbb4806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c034000 tx=0x7fbb5c02e000 comp rx=0 tx=0).stop 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 msgr2=0x7fbb6c1f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 --2- 172.21.15.43:0/3358151121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb6c071e60 0x7fbb6c1f5f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c410 tx=0x7fbb5400c8d0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb6a800640 1 -- 172.21.15.43:0/3358151121 reap_dead start 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 shutdown_connections 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 >> 172.21.15.43:0/3358151121 conn(0x7fbb6c0b2790 msgr2=0x7fbb6c072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 shutdown_connections 2024-04-26T20:32:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.443+0000 7fbb70e00640 1 -- 172.21.15.43:0/3358151121 wait complete. 2024-04-26T20:32:22.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T20:32:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.683+0000 7f3183000640 1 Processor -- start 2024-04-26T20:32:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.683+0000 7f3183000640 1 -- start start 2024-04-26T20:32:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.685+0000 7f3183000640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f317c0afb70 0x7f317c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.685+0000 7f3183000640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c072db0 con 0x7f317c0af710 2024-04-26T20:32:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.685+0000 7f3183000640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317c0cee40 con 0x7f317c15bb90 2024-04-26T20:32:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.685+0000 7f3183000640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f317c0cec10 con 0x7f317c0afb70 2024-04-26T20:32:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.686+0000 7f3181200640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f317c0af710 0x7f317c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59320/0 (socket says 172.21.15.43:59320) 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.686+0000 7f3181200640 1 -- 172.21.15.43:0/1267057685 learned_addr learned my addr 172.21.15.43:0/1267057685 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.686+0000 7f3182600640 1 --2- 172.21.15.43:0/1267057685 >> v2:172.21.15.43:3300/0 conn(0x7f317c0afb70 0x7f317c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.686+0000 7f317be00640 1 -- 172.21.15.43:0/1267057685 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557466817 0 0) 0x7f317c072db0 con 0x7f317c0af710 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.687+0000 7f3182600640 1 -- 172.21.15.43:0/1267057685 >> v1:172.21.15.164:6789/0 conn(0x7f317c15bb90 legacy=0x7f317c0737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.687+0000 7f3182600640 1 -- 172.21.15.43:0/1267057685 >> v1:172.21.15.43:6789/0 conn(0x7f317c0af710 legacy=0x7f317c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.687+0000 7f3182600640 1 -- 172.21.15.43:0/1267057685 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c0727f0 con 0x7f317c0afb70 2024-04-26T20:32:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.688+0000 7f3182600640 1 --2- 172.21.15.43:0/1267057685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c0afb70 0x7f317c0cfd10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3174009d90 tx=0x7f317402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9c7e54c5e5744d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.688+0000 7f317be00640 1 -- 172.21.15.43:0/1267057685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174036020 con 0x7f317c0afb70 2024-04-26T20:32:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.688+0000 7f317be00640 1 -- 172.21.15.43:0/1267057685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3174032800 con 0x7f317c0afb70 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.688+0000 7f317be00640 1 -- 172.21.15.43:0/1267057685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174032bd0 con 0x7f317c0afb70 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 -- 172.21.15.43:0/1267057685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c0afb70 msgr2=0x7f317c0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 --2- 172.21.15.43:0/1267057685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c0afb70 0x7f317c0cfd10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3174009d90 tx=0x7f317402c900 comp rx=0 tx=0).stop 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 -- 172.21.15.43:0/1267057685 shutdown_connections 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 --2- 172.21.15.43:0/1267057685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c0afb70 0x7f317c0cfd10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 -- 172.21.15.43:0/1267057685 >> 172.21.15.43:0/1267057685 conn(0x7f317c068090 msgr2=0x7f317c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 -- 172.21.15.43:0/1267057685 shutdown_connections 2024-04-26T20:32:22.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.689+0000 7f3183000640 1 -- 172.21.15.43:0/1267057685 wait complete. 2024-04-26T20:32:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.691+0000 7f3183000640 1 Processor -- start 2024-04-26T20:32:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.691+0000 7f3183000640 1 -- start start 2024-04-26T20:32:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 0x7f317c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f317c0afb70 0x7f317c15abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f317c073eb0 con 0x7f317c15bb90 2024-04-26T20:32:22.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f317c0725a0 con 0x7f317c0afb70 2024-04-26T20:32:22.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3183000640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f317c0cf0e0 con 0x7f317c0af710 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3182600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3182600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34728/0 (socket says 172.21.15.43:34728) 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3182600640 1 -- 172.21.15.43:0/607104590 learned_addr learned my addr 172.21.15.43:0/607104590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3181200640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f317c0afb70 0x7f317c15abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.692+0000 7f3181c00640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 0x7f317c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 -- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 msgr2=0x7f317c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 0x7f317c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 -- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f317c0afb70 msgr2=0x7f317c15abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f317c0afb70 0x7f317c15abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 -- 172.21.15.43:0/607104590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c156d70 con 0x7f317c15bb90 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3181c00640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 0x7f317c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3182600640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3174009eb0 tx=0x7f3174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174036040 con 0x7f317c15bb90 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.693+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317403dd00 con 0x7f317c15bb90 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.694+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174003bb0 con 0x7f317c15bb90 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.694+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c15c1c0 con 0x7f317c15bb90 2024-04-26T20:32:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.694+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c156a90 con 0x7f317c15bb90 2024-04-26T20:32:22.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.695+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c069350 con 0x7f317c15bb90 2024-04-26T20:32:22.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.696+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3174039020 con 0x7f317c15bb90 2024-04-26T20:32:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.697+0000 7f3179600640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 0x7f315c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.697+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f31740aee30 con 0x7f317c15bb90 2024-04-26T20:32:22.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.697+0000 7f3181c00640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 0x7f315c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.698+0000 7f3181c00640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 0x7f315c06ee00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3170007fd0 tx=0x7f3170009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:22.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.702+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3174042070 con 0x7f317c15bb90 2024-04-26T20:32:22.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.910+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f317c15c390 con 0x7f315c06c940 2024-04-26T20:32:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.911+0000 7f3179600640 1 -- 172.21.15.43:0/607104590 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f317c15c390 con 0x7f315c06c940 2024-04-26T20:32:22.912 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T20:32:22.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 msgr2=0x7f315c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 0x7f315c06ee00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3170007fd0 tx=0x7f3170009040 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 msgr2=0x7f317c1efe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3174009eb0 tx=0x7f3174004a40 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 shutdown_connections 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f315c06c940 0x7f315c06ee00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f317c15bb90 0x7f317c1efe80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f317c0afb70 0x7f317c15abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 --2- 172.21.15.43:0/607104590 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f317c0af710 0x7f317c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 >> 172.21.15.43:0/607104590 conn(0x7f317c068090 msgr2=0x7f317c0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 shutdown_connections 2024-04-26T20:32:22.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:22.914+0000 7f3183000640 1 -- 172.21.15.43:0/607104590 wait complete. 2024-04-26T20:32:22.933 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T20:32:22.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:32:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.150+0000 7f15efe00640 1 Processor -- start 2024-04-26T20:32:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.150+0000 7f15efe00640 1 -- start start 2024-04-26T20:32:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.151+0000 7f15efe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.151+0000 7f15efe00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f15f00d1c00 0x7f15f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.151+0000 7f15efe00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f00d2010 con 0x7f15f015f0a0 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.151+0000 7f15efe00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f15f00b4310 con 0x7f15f00d1c00 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.151+0000 7f15efe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f15f00d0f70 con 0x7f15f015f480 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.152+0000 7f15ee000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.152+0000 7f15ee000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.43:35306/0 (socket says 172.21.15.43:35306) 2024-04-26T20:32:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.152+0000 7f15ee000640 1 -- 172.21.15.43:0/3357752735 learned_addr learned my addr 172.21.15.43:0/3357752735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.152+0000 7f15ef400640 1 --2- 172.21.15.43:0/3357752735 >> v2:172.21.15.164:3300/0 conn(0x7f15f00d1c00 0x7f15f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.153+0000 7f15ee000640 1 -- 172.21.15.43:0/3357752735 >> v2:172.21.15.164:3300/0 conn(0x7f15f00d1c00 msgr2=0x7f15f00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.153+0000 7f15ee000640 1 --2- 172.21.15.43:0/3357752735 >> v2:172.21.15.164:3300/0 conn(0x7f15f00d1c00 0x7f15f00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.153+0000 7f15ee000640 1 -- 172.21.15.43:0/3357752735 >> v1:172.21.15.164:6789/0 conn(0x7f15f015f0a0 legacy=0x7f15f00d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.153+0000 7f15ed600640 1 -- 172.21.15.43:0/3357752735 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020265969 0 0) 0x7f15e0001490 con 0x7f15f015f0a0 2024-04-26T20:32:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.153+0000 7f15ee000640 1 -- 172.21.15.43:0/3357752735 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f00d1250 con 0x7f15f015f480 2024-04-26T20:32:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.154+0000 7f15ee000640 1 --2- 172.21.15.43:0/3357752735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009870 tx=0x7f15dc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b00e11d418b201 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.155+0000 7f15ed600640 1 -- 172.21.15.43:0/3357752735 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc037070 con 0x7f15f015f480 2024-04-26T20:32:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.155+0000 7f15ed600640 1 -- 172.21.15.43:0/3357752735 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc032440 con 0x7f15f015f480 2024-04-26T20:32:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.155+0000 7f15ed600640 1 -- 172.21.15.43:0/3357752735 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc032810 con 0x7f15f015f480 2024-04-26T20:32:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 -- 172.21.15.43:0/3357752735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 msgr2=0x7f15f00d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 --2- 172.21.15.43:0/3357752735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009870 tx=0x7f15dc02a270 comp rx=0 tx=0).stop 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 -- 172.21.15.43:0/3357752735 shutdown_connections 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 --2- 172.21.15.43:0/3357752735 >> v2:172.21.15.164:3300/0 conn(0x7f15f00d1c00 0x7f15f00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 --2- 172.21.15.43:0/3357752735 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f480 0x7f15f00d16c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 -- 172.21.15.43:0/3357752735 >> 172.21.15.43:0/3357752735 conn(0x7f15f009dff0 msgr2=0x7f15f00d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 -- 172.21.15.43:0/3357752735 shutdown_connections 2024-04-26T20:32:23.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.156+0000 7f15efe00640 1 -- 172.21.15.43:0/3357752735 wait complete. 2024-04-26T20:32:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.158+0000 7f15efe00640 1 Processor -- start 2024-04-26T20:32:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.158+0000 7f15efe00640 1 -- start start 2024-04-26T20:32:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f00d1c00 0x7f15f014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f0a0 0x7f15f006df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 0x7f15f006e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00d5af0 con 0x7f15f015f480 2024-04-26T20:32:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00b48b0 con 0x7f15f00d1c00 2024-04-26T20:32:23.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.159+0000 7f15efe00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f15f00d0b90 con 0x7f15f015f0a0 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 0x7f15f006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 0x7f15f006e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34736/0 (socket says 172.21.15.43:34736) 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 -- 172.21.15.43:0/1963042844 learned_addr learned my addr 172.21.15.43:0/1963042844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ee000640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f0a0 0x7f15f006df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15eea00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f00d1c00 0x7f15f014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 -- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f0a0 msgr2=0x7f15f006df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f15f015f0a0 0x7f15f006df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 -- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f00d1c00 msgr2=0x7f15f014f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f00d1c00 0x7f15f014f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f015f9f0 con 0x7f15f015f480 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15eea00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f00d1c00 0x7f15f014f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.160+0000 7f15ef400640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 0x7f15f006e480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f15e800e930 tx=0x7f15e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.161+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8017070 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.161+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e800c980 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.161+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f015fcd0 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.161+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e800cd70 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.161+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f0160290 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.162+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15f00ba8b0 con 0x7f15f015f480 2024-04-26T20:32:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.163+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15e801a070 con 0x7f15f015f480 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.164+0000 7f15e7400640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f15d006c920 0x7f15d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.164+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f15e808be00 con 0x7f15f015f480 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.164+0000 7f15e7400640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7f15d0072dc0 0x7f15d0075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.164+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15e8016ec0 con 0x7f15d0072dc0 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.164+0000 7f15eea00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f15d006c920 0x7f15d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.165+0000 7f15ee000640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7f15d0072dc0 0x7f15d0075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.165+0000 7f15eea00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f15d006c920 0x7f15d006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15e0001730 tx=0x7f15e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.166+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f15e8055f60 con 0x7f15f015f480 2024-04-26T20:32:23.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.167+0000 7f15ee000640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7f15d0072dc0 0x7f15d0075200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.169+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== osd.7 v2:172.21.15.164:6820/3685734532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15e8016ec0 con 0x7f15d0072dc0 2024-04-26T20:32:23.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.184+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 --> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f15f008dcd0 con 0x7f15d0072dc0 2024-04-26T20:32:23.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.185+0000 7f15e7400640 1 -- 172.21.15.43:0/1963042844 <== osd.7 v2:172.21.15.164:6820/3685734532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f15f008dcd0 con 0x7f15d0072dc0 2024-04-26T20:32:23.185 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-26T20:32:23.186 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:32:23.186 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.185+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7f15d0072dc0 msgr2=0x7f15d0075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.185+0000 7f15efe00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.164:6820/3685734532,v1:172.21.15.164:6822/3685734532] conn(0x7f15d0072dc0 0x7f15d0075200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f15d006c920 msgr2=0x7f15d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f15d006c920 0x7f15d006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f15e0001730 tx=0x7f15e0009040 comp rx=0 tx=0).stop 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 msgr2=0x7f15f006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 --2- 172.21.15.43:0/1963042844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15f015f480 0x7f15f006e480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f15e800e930 tx=0x7f15e800edf0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15ef400640 1 -- 172.21.15.43:0/1963042844 reap_dead start 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 shutdown_connections 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 >> 172.21.15.43:0/1963042844 conn(0x7f15f009dff0 msgr2=0x7f15f00b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 shutdown_connections 2024-04-26T20:32:23.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.186+0000 7f15efe00640 1 -- 172.21.15.43:0/1963042844 wait complete. 2024-04-26T20:32:23.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T20:32:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.416+0000 7f3e1de00640 1 Processor -- start 2024-04-26T20:32:23.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.417+0000 7f3e1de00640 1 -- start start 2024-04-26T20:32:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1de00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e180cd180 0x7f3e18072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1de00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e180ce480 0x7f3e1806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1de00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e180cd560 con 0x7f3e1815dc50 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1de00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3e18072d00 con 0x7f3e180ce480 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1de00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18072fe0 con 0x7f3e180cd180 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e1d400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e180ce480 0x7f3e1806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e17e00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3e1815dc50 0x7f3e180cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59334/0 (socket says 172.21.15.43:59334) 2024-04-26T20:32:23.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.418+0000 7f3e17e00640 1 -- 172.21.15.43:0/4220551273 learned_addr learned my addr 172.21.15.43:0/4220551273 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e1ca00640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e180cd180 0x7f3e18072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e1d400640 1 -- 172.21.15.43:0/4220551273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e180cd180 msgr2=0x7f3e18072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e1d400640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e180cd180 0x7f3e18072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e1d400640 1 -- 172.21.15.43:0/4220551273 >> v1:172.21.15.43:6789/0 conn(0x7f3e1815dc50 legacy=0x7f3e180cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e17400640 1 -- 172.21.15.43:0/4220551273 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658394929 0 0) 0x7f3e180cd560 con 0x7f3e1815dc50 2024-04-26T20:32:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.419+0000 7f3e1d400640 1 -- 172.21.15.43:0/4220551273 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e180732c0 con 0x7f3e180ce480 2024-04-26T20:32:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.421+0000 7f3e1d400640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ce480 0x7f3e1806dac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009d90 tx=0x7f3e1002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430654641d04dc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.421+0000 7f3e17400640 1 -- 172.21.15.43:0/4220551273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10007740 con 0x7f3e180ce480 2024-04-26T20:32:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.421+0000 7f3e17400640 1 -- 172.21.15.43:0/4220551273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e100078e0 con 0x7f3e180ce480 2024-04-26T20:32:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.421+0000 7f3e17400640 1 -- 172.21.15.43:0/4220551273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10007cb0 con 0x7f3e180ce480 2024-04-26T20:32:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 -- 172.21.15.43:0/4220551273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ce480 msgr2=0x7f3e1806dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ce480 0x7f3e1806dac0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009d90 tx=0x7f3e1002a660 comp rx=0 tx=0).stop 2024-04-26T20:32:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 -- 172.21.15.43:0/4220551273 shutdown_connections 2024-04-26T20:32:23.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180ce480 0x7f3e1806dac0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 --2- 172.21.15.43:0/4220551273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e180cd180 0x7f3e18072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 -- 172.21.15.43:0/4220551273 >> 172.21.15.43:0/4220551273 conn(0x7f3e18068090 msgr2=0x7f3e18068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 -- 172.21.15.43:0/4220551273 shutdown_connections 2024-04-26T20:32:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.422+0000 7f3e1de00640 1 -- 172.21.15.43:0/4220551273 wait complete. 2024-04-26T20:32:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.424+0000 7f3e1de00640 1 Processor -- start 2024-04-26T20:32:23.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.424+0000 7f3e1de00640 1 -- start start 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e180ce480 0x7f3e1812f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e1815dc50 0x7f3e18131090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1815e0d0 con 0x7f3e180cd180 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18073da0 con 0x7f3e180ce480 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1de00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1806e000 con 0x7f3e1815dc50 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34764/0 (socket says 172.21.15.43:34764) 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 -- 172.21.15.43:0/925522653 learned_addr learned my addr 172.21.15.43:0/925522653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 -- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e1815dc50 msgr2=0x7f3e18131090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e1815dc50 0x7f3e18131090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 -- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e180ce480 msgr2=0x7f3e1812f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e180ce480 0x7f3e1812f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.425+0000 7f3e1ca00640 1 -- 172.21.15.43:0/925522653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1812fff0 con 0x7f3e180cd180 2024-04-26T20:32:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e1ca00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00d800 tx=0x7f3e0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00b6c0 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c019070 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00ba80 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e181302d0 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.426+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e18130890 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.427+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e18069350 con 0x7f3e180cd180 2024-04-26T20:32:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.428+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e0c00bc20 con 0x7f3e180cd180 2024-04-26T20:32:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.429+0000 7f3e14c00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.429+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f3e0c08bee0 con 0x7f3e180cd180 2024-04-26T20:32:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.429+0000 7f3e17e00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.430+0000 7f3e17e00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 0x7f3df806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e04007fd0 tx=0x7f3e04007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.433+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e0c01c360 con 0x7f3e180cd180 2024-04-26T20:32:23.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.639+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f3e180d0670 con 0x7f3df806c8f0 2024-04-26T20:32:23.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.639+0000 7f3e14c00640 1 -- 172.21.15.43:0/925522653 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3e180d0670 con 0x7f3df806c8f0 2024-04-26T20:32:23.640 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 msgr2=0x7f3df806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 0x7f3df806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3e04007fd0 tx=0x7f3e04007960 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 msgr2=0x7f3e181e6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00d800 tx=0x7f3e0c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 shutdown_connections 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f3df806c8f0 0x7f3df806edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e1815dc50 0x7f3e18131090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e180ce480 0x7f3e1812f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 --2- 172.21.15.43:0/925522653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e180cd180 0x7f3e181e6610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 >> 172.21.15.43:0/925522653 conn(0x7f3e18068090 msgr2=0x7f3e180b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 shutdown_connections 2024-04-26T20:32:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.642+0000 7f3e1de00640 1 -- 172.21.15.43:0/925522653 wait complete. 2024-04-26T20:32:23.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:32:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.873+0000 7faab1e00640 1 Processor -- start 2024-04-26T20:32:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.873+0000 7faab1e00640 1 -- start start 2024-04-26T20:32:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faab1e00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faaac145590 0x7faaac0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faab1e00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faaac06f730 0x7faaac071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faab1e00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaac0cf060 con 0x7faaac0b3490 2024-04-26T20:32:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faab1e00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7faaac0d0100 con 0x7faaac145590 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faab1e00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7faaac0cf340 con 0x7faaac06f730 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faaaa000640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faaac0b3490 0x7faaac06f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59346/0 (socket says 172.21.15.43:59346) 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faaaa000640 1 -- 172.21.15.43:0/3302736717 learned_addr learned my addr 172.21.15.43:0/3302736717 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.875+0000 7faaaaa00640 1 --2- 172.21.15.43:0/3302736717 >> v2:172.21.15.43:3300/0 conn(0x7faaac145590 0x7faaac0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.876+0000 7faaaaa00640 1 -- 172.21.15.43:0/3302736717 >> v2:172.21.15.164:3300/0 conn(0x7faaac06f730 msgr2=0x7faaac071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.876+0000 7faaaaa00640 1 --2- 172.21.15.43:0/3302736717 >> v2:172.21.15.164:3300/0 conn(0x7faaac06f730 0x7faaac071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.876+0000 7faaaaa00640 1 -- 172.21.15.43:0/3302736717 >> v1:172.21.15.43:6789/0 conn(0x7faaac0b3490 legacy=0x7faaac06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.876+0000 7faaaaa00640 1 -- 172.21.15.43:0/3302736717 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac0cf620 con 0x7faaac145590 2024-04-26T20:32:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.877+0000 7faaaaa00640 1 --2- 172.21.15.43:0/3302736717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac145590 0x7faaac0ce780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faaac0d1b70 tx=0x7faa9c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3df113b24559ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.877+0000 7faaa9600640 1 -- 172.21.15.43:0/3302736717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c037070 con 0x7faaac145590 2024-04-26T20:32:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.877+0000 7faaa9600640 1 -- 172.21.15.43:0/3302736717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa9c032440 con 0x7faaac145590 2024-04-26T20:32:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.877+0000 7faaa9600640 1 -- 172.21.15.43:0/3302736717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c032810 con 0x7faaac145590 2024-04-26T20:32:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 -- 172.21.15.43:0/3302736717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac145590 msgr2=0x7faaac0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 --2- 172.21.15.43:0/3302736717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac145590 0x7faaac0ce780 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faaac0d1b70 tx=0x7faa9c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 -- 172.21.15.43:0/3302736717 shutdown_connections 2024-04-26T20:32:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 --2- 172.21.15.43:0/3302736717 >> v2:172.21.15.164:3300/0 conn(0x7faaac06f730 0x7faaac071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 --2- 172.21.15.43:0/3302736717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac145590 0x7faaac0ce780 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 -- 172.21.15.43:0/3302736717 >> 172.21.15.43:0/3302736717 conn(0x7faaac0b2900 msgr2=0x7faaac0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 -- 172.21.15.43:0/3302736717 shutdown_connections 2024-04-26T20:32:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.878+0000 7faab1e00640 1 -- 172.21.15.43:0/3302736717 wait complete. 2024-04-26T20:32:23.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.880+0000 7faab1e00640 1 Processor -- start 2024-04-26T20:32:23.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.880+0000 7faab1e00640 1 -- start start 2024-04-26T20:32:23.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaac06f730 0x7faaac15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faaac145590 0x7faaac1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faaac145970 con 0x7faaac0b3490 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faaac0cfbe0 con 0x7faaac06f730 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faab1e00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faaac0cecc0 con 0x7faaac145590 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34780/0 (socket says 172.21.15.43:34780) 2024-04-26T20:32:23.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 -- 172.21.15.43:0/3846714663 learned_addr learned my addr 172.21.15.43:0/3846714663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 -- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faaac145590 msgr2=0x7faaac1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faaac145590 0x7faaac1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 -- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaac06f730 msgr2=0x7faaac15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaac06f730 0x7faaac15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.881+0000 7faaaa000640 1 -- 172.21.15.43:0/3846714663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac1f56f0 con 0x7faaac0b3490 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faaaa000640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faa9800c440 tx=0x7faa98001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98017070 con 0x7faaac0b3490 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa98005ce0 con 0x7faaac0b3490 2024-04-26T20:32:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980072f0 con 0x7faaac0b3490 2024-04-26T20:32:23.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1f59d0 con 0x7faaac0b3490 2024-04-26T20:32:23.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.882+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac15d870 con 0x7faaac0b3490 2024-04-26T20:32:23.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.884+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac15da70 con 0x7faaac0b3490 2024-04-26T20:32:23.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.885+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa9801a070 con 0x7faaac0b3490 2024-04-26T20:32:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.885+0000 7faaa3400640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 0x7faa8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.886+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7faa9808acf0 con 0x7faaac0b3490 2024-04-26T20:32:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.886+0000 7faaaaa00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 0x7faa8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.887+0000 7faaaaa00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 0x7faa8c06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa9c034000 tx=0x7faa9c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:23.890+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faaac15da70 con 0x7faaac0b3490 2024-04-26T20:32:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.093+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faaac0ce330 con 0x7faa8c06c8f0 2024-04-26T20:32:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.095+0000 7faaa3400640 1 -- 172.21.15.43:0/3846714663 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13353 (secure 0 0 0) 0x7faaac0ce330 con 0x7faa8c06c8f0 2024-04-26T20:32:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 msgr2=0x7faa8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 0x7faa8c06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa9c034000 tx=0x7faa9c02c000 comp rx=0 tx=0).stop 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 msgr2=0x7faaac0d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faa9800c440 tx=0x7faa98001e70 comp rx=0 tx=0).stop 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 shutdown_connections 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7faa8c06c8f0 0x7faa8c06edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faaac145590 0x7faaac1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaac0b3490 0x7faaac0d04f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 --2- 172.21.15.43:0/3846714663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faaac06f730 0x7faaac15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 >> 172.21.15.43:0/3846714663 conn(0x7faaac0b2900 msgr2=0x7faaac0ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 shutdown_connections 2024-04-26T20:32:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:24.098+0000 7faab1e00640 1 -- 172.21.15.43:0/3846714663 wait complete. 2024-04-26T20:32:24.119 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-04-26T20:32:22.992040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237936,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736784,"statfs":{"total":773094113280,"available":772850466816,"internally_reserved":0,"allocated":5251072,"data_stored":2955352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.893281"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:32:14.127842+0000","last_change":"2024-04-26T20:25:52.404522+0000","last_active":"2024-04-26T20:32:14.127842+0000","last_peered":"2024-04-26T20:32:14.127842+0000","last_clean":"2024-04-26T20:32:14.127842+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:32:14.127842+0000","last_undegraded":"2024-04-26T20:32:14.127842+0000","last_fullsized":"2024-04-26T20:32:14.127842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:25:50.917018+0000","last_clean_scrub_stamp":"2024-04-26T20:25:50.917018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:20.595230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:32:24.122 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T20:25:50.917018+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=25, tm_sec=50, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=32, tm_sec=16, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T20:32:24.123 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T20:32:28.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:28.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:44.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:32:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.341+0000 7f5e5ac00640 1 Processor -- start 2024-04-26T20:32:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.342+0000 7f5e5ac00640 1 -- start start 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e5415eeb0 0x7f5e54163550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e5415f290 0x7f5e540726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f6e0 0x7f5e54072c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5e54160530 con 0x7f5e5415f290 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5e54163a90 con 0x7f5e5415eeb0 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.343+0000 7f5e5ac00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5e540af340 con 0x7f5e5415f6e0 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e5415f290 0x7f5e540726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e5415f290 0x7f5e540726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42568/0 (socket says 172.21.15.43:42568) 2024-04-26T20:32:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 -- 172.21.15.43:0/2310346858 learned_addr learned my addr 172.21.15.43:0/2310346858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53e00640 1 --2- 172.21.15.43:0/2310346858 >> v2:172.21.15.164:3300/0 conn(0x7f5e5415eeb0 0x7f5e54163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 -- 172.21.15.43:0/2310346858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f6e0 msgr2=0x7f5e54072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 --2- 172.21.15.43:0/2310346858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f6e0 0x7f5e54072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 -- 172.21.15.43:0/2310346858 >> v2:172.21.15.164:3300/0 conn(0x7f5e5415eeb0 msgr2=0x7f5e54163550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 --2- 172.21.15.43:0/2310346858 >> v2:172.21.15.164:3300/0 conn(0x7f5e5415eeb0 0x7f5e54163550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.344+0000 7f5e53400640 1 -- 172.21.15.43:0/2310346858 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5415fc90 con 0x7f5e5415f290 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.345+0000 7f5e53400640 1 --2- 172.21.15.43:0/2310346858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415f290 0x7f5e540726c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5e540b68e0 tx=0x7f5e4002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5cc4c9e9859865 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e52a00640 1 -- 172.21.15.43:0/2310346858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40036070 con 0x7f5e5415f290 2024-04-26T20:32:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e52a00640 1 -- 172.21.15.43:0/2310346858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e40031600 con 0x7f5e5415f290 2024-04-26T20:32:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e52a00640 1 -- 172.21.15.43:0/2310346858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400319d0 con 0x7f5e5415f290 2024-04-26T20:32:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 -- 172.21.15.43:0/2310346858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415f290 msgr2=0x7f5e540726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/2310346858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415f290 0x7f5e540726c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5e540b68e0 tx=0x7f5e4002a6d0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 -- 172.21.15.43:0/2310346858 shutdown_connections 2024-04-26T20:32:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/2310346858 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f6e0 0x7f5e54072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/2310346858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415f290 0x7f5e540726c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/2310346858 >> v2:172.21.15.164:3300/0 conn(0x7f5e5415eeb0 0x7f5e54163550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.346+0000 7f5e5ac00640 1 -- 172.21.15.43:0/2310346858 >> 172.21.15.43:0/2310346858 conn(0x7f5e5406c350 msgr2=0x7f5e5406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.347+0000 7f5e5ac00640 1 -- 172.21.15.43:0/2310346858 shutdown_connections 2024-04-26T20:32:44.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.347+0000 7f5e5ac00640 1 -- 172.21.15.43:0/2310346858 wait complete. 2024-04-26T20:32:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.348+0000 7f5e5ac00640 1 Processor -- start 2024-04-26T20:32:44.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 -- start start 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f290 0x7f5e541e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e5415f6e0 0x7f5e541e9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e540af5a0 con 0x7f5e5415eeb0 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54160250 con 0x7f5e5415f6e0 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.349+0000 7f5e5ac00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5e540b5030 con 0x7f5e5415f290 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42582/0 (socket says 172.21.15.43:42582) 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 -- 172.21.15.43:0/3671583157 learned_addr learned my addr 172.21.15.43:0/3671583157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 -- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f290 msgr2=0x7f5e541e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f290 0x7f5e541e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 -- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e5415f6e0 msgr2=0x7f5e541e9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e5415f6e0 0x7f5e541e9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 -- 172.21.15.43:0/3671583157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e54159140 con 0x7f5e5415eeb0 2024-04-26T20:32:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.350+0000 7f5e53e00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d830 tx=0x7f5e4400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.351+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400b6c0 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.351+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e44019070 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.351+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400ba80 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.351+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e54159420 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.351+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541599e0 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.352+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e54069350 con 0x7f5e5415eeb0 2024-04-26T20:32:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.353+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e4401e460 con 0x7f5e5415eeb0 2024-04-26T20:32:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.354+0000 7f5e4be00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 0x7f5e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.355+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f5e4408c160 con 0x7f5e5415eeb0 2024-04-26T20:32:44.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.355+0000 7f5e53400640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 0x7f5e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.355+0000 7f5e53400640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 0x7f5e3406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e40033000 tx=0x7f5e4002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.359+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e4405a660 con 0x7f5e5415eeb0 2024-04-26T20:32:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.558+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 --> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e540d0e70 con 0x7f5e3406c8f0 2024-04-26T20:32:44.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.560+0000 7f5e4be00640 1 -- 172.21.15.43:0/3671583157 <== mgr.4107 v2:172.21.15.43:6832/1074533441 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7f5e540d0e70 con 0x7f5e3406c8f0 2024-04-26T20:32:44.561 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:44.561 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.562+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 msgr2=0x7f5e3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.562+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 0x7f5e3406edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e40033000 tx=0x7f5e4002e000 comp rx=0 tx=0).stop 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.562+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 msgr2=0x7f5e541506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.562+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d830 tx=0x7f5e4400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.562+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 shutdown_connections 2024-04-26T20:32:44.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f5e3406c8f0 0x7f5e3406edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e5415f290 0x7f5e541e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e5415f6e0 0x7f5e541e9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 --2- 172.21.15.43:0/3671583157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e5415eeb0 0x7f5e541506c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 >> 172.21.15.43:0/3671583157 conn(0x7f5e5406c350 msgr2=0x7f5e540726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 shutdown_connections 2024-04-26T20:32:44.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.563+0000 7f5e5ac00640 1 -- 172.21.15.43:0/3671583157 wait complete. 2024-04-26T20:32:44.583 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-04-26T20:32:42.997813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237928,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736792,"statfs":{"total":773094113280,"available":772850475008,"internally_reserved":0,"allocated":5251072,"data_stored":2955352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003409"},"pg_stats":[{"pgid":"1.0","version":"43'119","reported_seq":225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:32:24.283339+0000","last_change":"2024-04-26T20:32:24.283442+0000","last_active":"2024-04-26T20:32:24.283339+0000","last_peered":"2024-04-26T20:32:24.283339+0000","last_clean":"2024-04-26T20:32:24.283339+0000","last_became_active":"2024-04-26T20:25:52.401878+0000","last_became_peered":"2024-04-26T20:25:52.401878+0000","last_unstale":"2024-04-26T20:32:24.283339+0000","last_undegraded":"2024-04-26T20:32:24.283339+0000","last_fullsized":"2024-04-26T20:32:24.283339+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-04-26T20:32:24.283161+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-04-26T20:32:24.283161+0000","last_clean_scrub_stamp":"2024-04-26T20:32:24.283161+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:12.311499+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":507904,"data_stored":221827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1101824,"data_stored":812195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:32:44.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T20:32:44.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.783+0000 7fdd7e400640 1 Processor -- start 2024-04-26T20:32:44.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.783+0000 7fdd7e400640 1 -- start start 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd7e400640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdd7815f180 0x7fdd780d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd7e400640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd780d2d70 con 0x7fdd780b3cb0 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd7e400640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd780d06e0 con 0x7fdd780d3830 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd7e400640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7815f560 con 0x7fdd7815f180 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd6fe00640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdd780b3cb0 0x7fdd780d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53758/0 (socket says 172.21.15.43:53758) 2024-04-26T20:32:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.785+0000 7fdd6fe00640 1 -- 172.21.15.43:0/2693727317 learned_addr learned my addr 172.21.15.43:0/2693727317 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:44.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd7d000640 1 --2- 172.21.15.43:0/2693727317 >> v2:172.21.15.164:3300/0 conn(0x7fdd7815f180 0x7fdd780d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271747594 0 0) 0x7fdd780d2d70 con 0x7fdd780b3cb0 2024-04-26T20:32:44.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd5c003520 con 0x7fdd780b3cb0 2024-04-26T20:32:44.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240399795 0 0) 0x7fdd5c003520 con 0x7fdd780b3cb0 2024-04-26T20:32:44.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 >> v2:172.21.15.164:3300/0 conn(0x7fdd7815f180 msgr2=0x7fdd780d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 --2- 172.21.15.43:0/2693727317 >> v2:172.21.15.164:3300/0 conn(0x7fdd7815f180 0x7fdd780d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 >> v1:172.21.15.164:6789/0 conn(0x7fdd780d3830 legacy=0x7fdd780b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd780d09c0 con 0x7fdd780b3cb0 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 612266112 0 0) 0x7fdd60001690 con 0x7fdd780b3cb0 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 >> v1:172.21.15.43:6789/0 conn(0x7fdd780b3cb0 legacy=0x7fdd780d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c003a20 0x7fdd5c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c004800 0x7fdd5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 0x7fdd5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7815f560 con 0x7fdd5c0051c0 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00c0f0 con 0x7fdd5c003a20 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.786+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5c00c2f0 con 0x7fdd5c004800 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 0x7fdd5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd6fe00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c004800 0x7fdd5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7d000640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c003a20 0x7fdd5c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 -- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c004800 msgr2=0x7fdd5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c004800 0x7fdd5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 -- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c003a20 msgr2=0x7fdd5c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd5c003a20 0x7fdd5c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.787+0000 7fdd7da00640 1 -- 172.21.15.43:0/2693727317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c003520 con 0x7fdd5c0051c0 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.788+0000 7fdd7da00640 1 -- 172.21.15.43:0/2693727317 reap_dead start 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 0x7fdd5c00b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdd7808d330 tx=0x7fdd5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea38db2cccc5f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70001a70 con 0x7fdd5c0051c0 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7002add0 con 0x7fdd5c0051c0 2024-04-26T20:32:44.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd77e00640 1 -- 172.21.15.43:0/2693727317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003bb0 con 0x7fdd5c0051c0 2024-04-26T20:32:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd7e400640 1 -- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 msgr2=0x7fdd5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.789+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 0x7fdd5c00b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdd7808d330 tx=0x7fdd5c005830 comp rx=0 tx=0).stop 2024-04-26T20:32:44.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.790+0000 7fdd7e400640 1 -- 172.21.15.43:0/2693727317 shutdown_connections 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.790+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2693727317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd5c0051c0 0x7fdd5c00b9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.790+0000 7fdd7e400640 1 -- 172.21.15.43:0/2693727317 >> 172.21.15.43:0/2693727317 conn(0x7fdd780d4bd0 msgr2=0x7fdd780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.790+0000 7fdd7e400640 1 -- 172.21.15.43:0/2693727317 shutdown_connections 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.790+0000 7fdd7e400640 1 -- 172.21.15.43:0/2693727317 wait complete. 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.791+0000 7fdd7e400640 1 Processor -- start 2024-04-26T20:32:44.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.792+0000 7fdd7e400640 1 -- start start 2024-04-26T20:32:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c0051c0 0x7fdd780b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd7815a7e0 0x7fdd780b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780d11c0 con 0x7fdd780b2e20 2024-04-26T20:32:44.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdd780d2a40 con 0x7fdd7815a7e0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7e400640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdd781f5740 con 0x7fdd5c0051c0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42588/0 (socket says 172.21.15.43:42588) 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 -- 172.21.15.43:0/2100475568 learned_addr learned my addr 172.21.15.43:0/2100475568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd6fe00640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd7815a7e0 0x7fdd780b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 -- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c0051c0 msgr2=0x7fdd780b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7d000640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c0051c0 0x7fdd780b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c0051c0 0x7fdd780b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 -- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd7815a7e0 msgr2=0x7fdd780b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd7815a7e0 0x7fdd780b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.793+0000 7fdd7da00640 1 -- 172.21.15.43:0/2100475568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781f5460 con 0x7fdd780b2e20 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd7da00640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdd70001ee0 tx=0x7fdd700014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007db0 con 0x7fdd780b2e20 2024-04-26T20:32:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7002abd0 con 0x7fdd780b2e20 2024-04-26T20:32:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70005380 con 0x7fdd780b2e20 2024-04-26T20:32:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd780b52f0 con 0x7fdd780b2e20 2024-04-26T20:32:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.794+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd780b58b0 con 0x7fdd780b2e20 2024-04-26T20:32:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.796+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd780506c0 con 0x7fdd780b2e20 2024-04-26T20:32:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.797+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd70005520 con 0x7fdd780b2e20 2024-04-26T20:32:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.798+0000 7fdd75600640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 0x7fdd5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.798+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fdd700aa980 con 0x7fdd780b2e20 2024-04-26T20:32:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.799+0000 7fdd7d000640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 0x7fdd5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:44.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.799+0000 7fdd7d000640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 0x7fdd5406ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098d0 tx=0x7fdd68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:44.803+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd70045070 con 0x7fdd780b2e20 2024-04-26T20:32:45.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.007+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdd780b5a80 con 0x7fdd780b2e20 2024-04-26T20:32:45.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.013+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fdd70078f00 con 0x7fdd780b2e20 2024-04-26T20:32:45.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.014+0000 7fdd75600640 1 -- 172.21.15.43:0/2100475568 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdd700b0080 con 0x7fdd780b2e20 2024-04-26T20:32:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 msgr2=0x7fdd5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 0x7fdd5406ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098d0 tx=0x7fdd68008040 comp rx=0 tx=0).stop 2024-04-26T20:32:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 msgr2=0x7fdd781f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdd70001ee0 tx=0x7fdd700014a0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 shutdown_connections 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7fdd5406c920 0x7fdd5406ede0 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098d0 tx=0x7fdd68008040 comp rx=0 tx=0).stop 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd780b2e20 0x7fdd781f4730 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdd7815a7e0 0x7fdd780b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 --2- 172.21.15.43:0/2100475568 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdd5c0051c0 0x7fdd780b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 >> 172.21.15.43:0/2100475568 conn(0x7fdd780d4bd0 msgr2=0x7fdd780d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 shutdown_connections 2024-04-26T20:32:45.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.017+0000 7fdd7e400640 1 -- 172.21.15.43:0/2100475568 wait complete. 2024-04-26T20:32:45.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:32:45.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:32:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.251+0000 7f2cb1a00640 1 Processor -- start 2024-04-26T20:32:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.251+0000 7f2cb1a00640 1 -- start start 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.253+0000 7f2cb1a00640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2cac06ca60 0x7f2cac0b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.253+0000 7f2cb1a00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac15fbe0 con 0x7f2cac15f620 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.253+0000 7f2cb1a00640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac15fec0 con 0x7f2cac148950 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.253+0000 7f2cb1a00640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0b51b0 con 0x7f2cac06ca60 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2cab400640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2cac15f620 0x7f2cac0b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53772/0 (socket says 172.21.15.43:53772) 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2cab400640 1 -- 172.21.15.43:0/690268476 learned_addr learned my addr 172.21.15.43:0/690268476 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-26T20:32:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2ca3e00640 1 --2- 172.21.15.43:0/690268476 >> v2:172.21.15.43:3300/0 conn(0x7f2cac06ca60 0x7f2cac0b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:45.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2ca3e00640 1 -- 172.21.15.43:0/690268476 >> v1:172.21.15.164:6789/0 conn(0x7f2cac148950 legacy=0x7f2cac06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:32:45.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2ca3e00640 1 -- 172.21.15.43:0/690268476 >> v1:172.21.15.43:6789/0 conn(0x7f2cac15f620 legacy=0x7f2cac0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2caa000640 1 -- 172.21.15.43:0/690268476 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66808384 0 0) 0x7f2cac15fbe0 con 0x7f2cac15f620 2024-04-26T20:32:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.254+0000 7f2ca3e00640 1 -- 172.21.15.43:0/690268476 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1601a0 con 0x7f2cac06ca60 2024-04-26T20:32:45.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.255+0000 7f2ca3e00640 1 --2- 172.21.15.43:0/690268476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac06ca60 0x7f2cac0b0e60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2c940097a0 tx=0x7f2c9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1b9a5c15fb75e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2caa000640 1 -- 172.21.15.43:0/690268476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94037070 con 0x7f2cac06ca60 2024-04-26T20:32:45.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2caa000640 1 -- 172.21.15.43:0/690268476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2c94032440 con 0x7f2cac06ca60 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2caa000640 1 -- 172.21.15.43:0/690268476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94032830 con 0x7f2cac06ca60 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 -- 172.21.15.43:0/690268476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac06ca60 msgr2=0x7f2cac0b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/690268476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac06ca60 0x7f2cac0b0e60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2c940097a0 tx=0x7f2c9402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 -- 172.21.15.43:0/690268476 shutdown_connections 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/690268476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac06ca60 0x7f2cac0b0e60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 -- 172.21.15.43:0/690268476 >> 172.21.15.43:0/690268476 conn(0x7f2cac0b39b0 msgr2=0x7f2cac0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.256+0000 7f2cb1a00640 1 -- 172.21.15.43:0/690268476 shutdown_connections 2024-04-26T20:32:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.257+0000 7f2cb1a00640 1 -- 172.21.15.43:0/690268476 wait complete. 2024-04-26T20:32:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.258+0000 7f2cb1a00640 1 Processor -- start 2024-04-26T20:32:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.258+0000 7f2cb1a00640 1 -- start start 2024-04-26T20:32:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 0x7f2cac15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2cac15f620 0x7f2cac1e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0b4710 con 0x7f2cac148950 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac148d30 con 0x7f2cac06ca60 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2cb1a00640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac0b4890 con 0x7f2cac15f620 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.259+0000 7f2ca3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42610/0 (socket says 172.21.15.43:42610) 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 -- 172.21.15.43:0/4137767664 learned_addr learned my addr 172.21.15.43:0/4137767664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-26T20:32:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2caaa00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 0x7f2cac15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 -- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2cac15f620 msgr2=0x7f2cac1e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2cac15f620 0x7f2cac1e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 -- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 msgr2=0x7f2cac15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 0x7f2cac15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 -- 172.21.15.43:0/4137767664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac15d060 con 0x7f2cac148950 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2caaa00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 0x7f2cac15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.260+0000 7f2ca3e00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2c94035040 tx=0x7f2c940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:45.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.261+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94037040 con 0x7f2cac148950 2024-04-26T20:32:45.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.261+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2c94030830 con 0x7f2cac148950 2024-04-26T20:32:45.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.261+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94030c40 con 0x7f2cac148950 2024-04-26T20:32:45.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.261+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac0b1e70 con 0x7f2cac148950 2024-04-26T20:32:45.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.261+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac0b2430 con 0x7f2cac148950 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.262+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac0b4fd0 con 0x7f2cac148950 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.263+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c94044020 con 0x7f2cac148950 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.264+0000 7f2ca2a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.264+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f2c940abfb0 con 0x7f2cac148950 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.264+0000 7f2caaa00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:32:45.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.265+0000 7f2caaa00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003eb0 tx=0x7f2c9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:32:45.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.268+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cac0b4fd0 con 0x7f2cac148950 2024-04-26T20:32:45.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.513+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cac069a60 con 0x7f2cac148950 2024-04-26T20:32:45.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.514+0000 7f2ca2a00640 1 -- 172.21.15.43:0/4137767664 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3875 (secure 0 0 0) 0x7f2c9407a4b0 con 0x7f2cac148950 2024-04-26T20:32:45.515 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-26T20:32:45.515 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5062,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/1044910699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1044910699},{"type":"v1","addr":"172.21.15.43:6836","nonce":1044910699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/1288399860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":1288399860},{"type":"v1","addr":"172.21.15.164:6834","nonce":1288399860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3032290339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":3032290339},{"type":"v1","addr":"172.21.15.164:6835","nonce":3032290339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5829,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/1559991146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1559991146},{"type":"v1","addr":"172.21.15.43:6837","nonce":1559991146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-26T20:32:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 20 2024-04-26T20:32:45.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.516+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 msgr2=0x7f2c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.516+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 0x7f2c8806edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c003eb0 tx=0x7f2c9c009040 comp rx=0 tx=0).stop 2024-04-26T20:32:45.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.516+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 msgr2=0x7f2cac0c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.516+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2c94035040 tx=0x7f2c940046c0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 shutdown_connections 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:6832/1074533441,v1:172.21.15.43:6833/1074533441] conn(0x7f2c8806c8f0 0x7f2c8806edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2cac15f620 0x7f2cac1e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cac148950 0x7f2cac0c6c20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 --2- 172.21.15.43:0/4137767664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2cac06ca60 0x7f2cac15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 >> 172.21.15.43:0/4137767664 conn(0x7f2cac0b39b0 msgr2=0x7f2cac0b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 shutdown_connections 2024-04-26T20:32:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-26T20:32:45.517+0000 7f2cb1a00640 1 -- 172.21.15.43:0/4137767664 wait complete. 2024-04-26T20:32:45.537 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T20:32:45.537 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T20:32:45.537 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:32:45.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.556 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:32:45.556 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:32:45.556 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.629 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:32:45.629 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:32:45.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.661 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:32:45.661 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:32:45.661 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.728 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:32:45.729 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T20:32:45.729 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T20:32:45.729 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.799 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T20:32:45.799 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T20:32:45.799 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.894 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T20:32:45.895 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T20:32:45.895 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:45.973 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T20:32:45.973 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T20:32:45.973 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.044 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T20:32:46.044 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T20:32:46.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.095 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T20:32:46.095 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T20:32:46.095 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.179 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T20:32:46.179 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T20:32:46.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.245 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T20:32:46.245 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T20:32:46.246 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.289 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T20:32:46.290 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T20:32:46.290 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T20:32:46.290 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:46.382 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T20:32:46.382 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T20:32:46.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:50.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-26T20:32:52.484 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T20:32:52.485 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T20:32:52.485 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T20:32:52.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:52.522 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T20:32:52.523 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T20:32:52.523 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:52.563 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T20:32:52.563 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T20:32:52.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:32:52.632 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T20:32:52.632 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T20:32:52.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-04-26T20:32:52.742 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:32:52.743 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T20:32:52.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:32:52.974 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T20:32:53.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:32:53.145 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T20:32:53.315 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:32:53.315 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T20:32:53.442 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:32:53.442 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T20:32:53.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:32:53.707 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T20:32:53.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:32:53.848 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T20:32:53.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:32:53.986 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T20:32:54.132 INFO:tasks.ceph:Archiving mon data... 2024-04-26T20:32:54.134 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/data/mon.a.tgz 2024-04-26T20:32:54.134 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-04-26T20:32:54.195 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.18i4hUpaav 2024-04-26T20:32:54.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.18i4hUpaav 2024-04-26T20:32:54.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.18i4hUpaav 2024-04-26T20:32:54.530 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.18i4hUpaav is 353KB 2024-04-26T20:32:54.604 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.18i4hUpaav 2024-04-26T20:32:54.661 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/data/mon.b.tgz 2024-04-26T20:32:54.661 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-26T20:32:54.719 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.c3Aydy1h4A 2024-04-26T20:32:54.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.c3Aydy1h4A 2024-04-26T20:32:54.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.c3Aydy1h4A 2024-04-26T20:32:55.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-26T20:32:55.061 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.c3Aydy1h4A is 467KB 2024-04-26T20:32:55.232 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.c3Aydy1h4A 2024-04-26T20:32:55.301 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/data/mon.c.tgz 2024-04-26T20:32:55.302 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-26T20:32:55.364 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.or5sf22LkY 2024-04-26T20:32:55.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.or5sf22LkY 2024-04-26T20:32:55.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.or5sf22LkY 2024-04-26T20:32:55.767 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.or5sf22LkY is 467KB 2024-04-26T20:32:56.076 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.or5sf22LkY 2024-04-26T20:32:56.131 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T20:32:56.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:32:56.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:32:56.287 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:32:56.350 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:32:56.351 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T20:32:56.405 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi043/crash 2024-04-26T20:32:56.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:32:56.503 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi164/crash 2024-04-26T20:32:56.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:32:56.586 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T20:32:56.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:56.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:56.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:56.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:32:56.893 INFO:tasks.ceph:Compressing logs... 2024-04-26T20:32:56.893 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:32:56.901 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:32:56.978 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33821.log 2024-04-26T20:32:56.979 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:32:56.980 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39677.log 2024-04-26T20:32:56.980 INFO:teuthology.orchestra.run.smithi164.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33821.log.gz 2024-04-26T20:32:56.981 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:32:56.982 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.3.39677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-04-26T20:32:56.983 INFO:teuthology.orchestra.run.smithi164.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:32:56.983 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/ceph-client.3.39677.log.gz 2024-04-26T20:32:56.983 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T20:32:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38033.log 2024-04-26T20:32:56.984 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33610.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T20:32:56.984 INFO:teuthology.orchestra.run.smithi164.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-04-26T20:32:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-04-26T20:32:56.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T20:32:56.985 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41299.log 2024-04-26T20:32:56.987 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38033.log.gz 2024-04-26T20:32:56.987 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33570.log 2024-04-26T20:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42954.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-04-26T20:32:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40166.log 2024-04-26T20:32:56.988 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33701.log 2024-04-26T20:32:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106582.log 2024-04-26T20:32:56.989 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33570.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T20:32:56.990 INFO:teuthology.orchestra.run.smithi164.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33570.log.gz 2024-04-26T20:32:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40166.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41299.log.gz 2024-04-26T20:32:56.991 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40166.log.gz 2024-04-26T20:32:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44633.log 2024-04-26T20:32:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33701.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33701.log.gz 2024-04-26T20:32:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33741.log 2024-04-26T20:32:56.992 INFO:teuthology.orchestra.run.smithi164.stderr:gzip/var/log/ceph/ceph-osd.4.log: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T20:32:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37591.log 2024-04-26T20:32:56.993 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106582.log.gz 2024-04-26T20:32:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44633.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.44633.log.gz --verbose 2024-04-26T20:32:56.993 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.38075.log 2024-04-26T20:32:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33741.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33741.log.gz 2024-04-26T20:32:56.994 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T20:32:56.995 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T20:32:56.995 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-04-26T20:32:56.995 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37591.log.gz 2024-04-26T20:32:56.996 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-04-26T20:32:56.996 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38075.log.gz 2024-04-26T20:32:56.996 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33530.log 2024-04-26T20:32:56.997 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48142.log 2024-04-26T20:32:56.997 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-04-26T20:32:56.997 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45851.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-04-26T20:32:56.997 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43429.log 2024-04-26T20:32:56.999 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2024-04-26T20:32:56.999 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48142.log.gz 2024-04-26T20:32:56.999 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-04-26T20:32:57.000 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39221.log 2024-04-26T20:32:57.000 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43429.log.gz 2024-04-26T20:32:57.000 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39997.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2024-04-26T20:32:57.000 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39769.log 2024-04-26T20:32:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106229.log 2024-04-26T20:32:57.002 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-04-26T20:32:57.003 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48756.log 2024-04-26T20:32:57.003 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39769.log.gz 2024-04-26T20:32:57.003 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.1.39221.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33530.log.gz 2024-04-26T20:32:57.003 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33781.log 2024-04-26T20:32:57.004 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T20:32:57.004 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37535.log 2024-04-26T20:32:57.005 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106229.log.gz 2024-04-26T20:32:57.005 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.48756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105586.log 2024-04-26T20:32:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37535.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37535.log.gz 2024-04-26T20:32:57.006 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-04-26T20:32:57.006 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2024-04-26T20:32:57.006 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33781.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33781.log.gz 2024-04-26T20:32:57.008 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105159.log 2024-04-26T20:32:57.008 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105586.log.gz 2024-04-26T20:32:57.008 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45373.log 2024-04-26T20:32:57.008 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-04-26T20:32:57.009 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33660.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2024-04-26T20:32:57.009 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T20:32:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-04-26T20:32:57.010 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105159.log.gz 2024-04-26T20:32:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45373.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45373.log.gz 2024-04-26T20:32:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47802.log 2024-04-26T20:32:57.012 INFO:teuthology.orchestra.run.smithi164.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.1.39221.log.gz 2024-04-26T20:32:57.012 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T20:32:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-04-26T20:32:57.012 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-04-26T20:32:57.013 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47802.log: 95.5% -- replaced with /var/log/ceph/ceph-client.0.48756.log.gz 2024-04-26T20:32:57.013 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48478.log 2024-04-26T20:32:57.013 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47802.log.gz 2024-04-26T20:32:57.013 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106540.log 2024-04-26T20:32:57.013 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T20:32:57.014 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-04-26T20:32:57.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48478.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48478.log.gz 2024-04-26T20:32:57.014 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41257.log 2024-04-26T20:32:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45583.log 2024-04-26T20:32:57.015 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106540.log.gz 2024-04-26T20:32:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2024-04-26T20:32:57.016 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41257.log.gz 2024-04-26T20:32:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-04-26T20:32:57.017 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45583.log.gz 2024-04-26T20:32:57.018 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41467.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2024-04-26T20:32:57.018 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-04-26T20:32:57.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46325.log 2024-04-26T20:32:57.022 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-04-26T20:32:57.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40178.log 2024-04-26T20:32:57.022 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-04-26T20:32:57.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46325.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46325.log.gz --verbose 2024-04-26T20:32:57.022 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.46997.log 2024-04-26T20:32:57.023 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45725.log 2024-04-26T20:32:57.023 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40178.log.gz 2024-04-26T20:32:57.023 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 2024-04-26T20:32:57.024 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-04-26T20:32:57.024 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 89.2% 2024-04-26T20:32:57.024 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45725.log.gz 2024-04-26T20:32:57.025 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45415.log 2024-04-26T20:32:57.025 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-04-26T20:32:57.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-04-26T20:32:57.026 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph.log 2024-04-26T20:32:57.027 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48058.log 2024-04-26T20:32:57.027 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T20:32:57.028 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45415.log.gz 2024-04-26T20:32:57.028 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104991.log 2024-04-26T20:32:57.029 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:32:57.029 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-04-26T20:32:57.029 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48058.log.gz 2024-04-26T20:32:57.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45641.log 2024-04-26T20:32:57.030 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104991.log.gz 2024-04-26T20:32:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44549.log 2024-04-26T20:32:57.031 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-04-26T20:32:57.032 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106145.log 2024-04-26T20:32:57.032 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45641.log.gz 2024-04-26T20:32:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2024-04-26T20:32:57.033 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44549.log.gz 2024-04-26T20:32:57.034 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-04-26T20:32:57.034 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106145.log.gz 2024-04-26T20:32:57.035 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2024-04-26T20:32:57.035 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2024-04-26T20:32:57.036 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41889.log 2024-04-26T20:32:57.036 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-04-26T20:32:57.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41635.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T20:32:57.037 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2024-04-26T20:32:57.039 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-04-26T20:32:57.039 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41889.log.gz 2024-04-26T20:32:57.039 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-04-26T20:32:57.041 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48412.log 2024-04-26T20:32:57.041 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-04-26T20:32:57.042 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48100.log 2024-04-26T20:32:57.042 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-04-26T20:32:57.043 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105346.log 2024-04-26T20:32:57.043 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48412.log.gz 2024-04-26T20:32:57.044 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37799.log 2024-04-26T20:32:57.044 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2024-04-26T20:32:57.045 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39955.log 2024-04-26T20:32:57.045 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105346.log.gz 2024-04-26T20:32:57.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47621.log 2024-04-26T20:32:57.047 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37799.log.gz 2024-04-26T20:32:57.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46871.log 2024-04-26T20:32:57.048 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39955.log.gz 2024-04-26T20:32:57.049 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44449.log 2024-04-26T20:32:57.049 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47621.log.gz 2024-04-26T20:32:57.050 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-04-26T20:32:57.050 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46871.log.gz 2024-04-26T20:32:57.051 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38243.log 2024-04-26T20:32:57.051 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44449.log.gz 2024-04-26T20:32:57.052 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46619.log 2024-04-26T20:32:57.052 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-26T20:32:57.053 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-04-26T20:32:57.053 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38243.log.gz 2024-04-26T20:32:57.054 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47081.log 2024-04-26T20:32:57.055 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46619.log.gz 2024-04-26T20:32:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-04-26T20:32:57.055 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-04-26T20:32:57.056 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47081.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:32:57.056 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47081.log.gz 2024-04-26T20:32:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33308.log 2024-04-26T20:32:57.057 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-04-26T20:32:57.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-04-26T20:32:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33576.log 2024-04-26T20:32:57.060 INFO:teuthology.orchestra.run.smithi043.stderr: 90.1% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33308.log.gz -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:32:57.060 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-04-26T20:32:57.061 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-04-26T20:32:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39439.log 2024-04-26T20:32:57.062 INFO:teuthology.orchestra.run.smithi043.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33576.log.gz 2024-04-26T20:32:57.063 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-04-26T20:32:57.063 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-04-26T20:32:57.064 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37358.log 2024-04-26T20:32:57.064 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39439.log.gz 2024-04-26T20:32:57.065 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41551.log 2024-04-26T20:32:57.065 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-04-26T20:32:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37358.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37358.log.gz 2024-04-26T20:32:57.066 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-04-26T20:32:57.067 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-04-26T20:32:57.067 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41551.log.gz 2024-04-26T20:32:57.068 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105430.log 2024-04-26T20:32:57.068 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-04-26T20:32:57.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2024-04-26T20:32:57.069 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-04-26T20:32:57.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-04-26T20:32:57.070 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105430.log.gz 2024-04-26T20:32:57.071 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37454.log 2024-04-26T20:32:57.072 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2024-04-26T20:32:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46493.log 2024-04-26T20:32:57.073 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-04-26T20:32:57.074 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2024-04-26T20:32:57.074 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37454.log.gz 2024-04-26T20:32:57.074 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46493.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T20:32:57.075 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46493.log.gz 2024-04-26T20:32:57.075 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43064.log 2024-04-26T20:32:57.076 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2024-04-26T20:32:57.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-04-26T20:32:57.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-04-26T20:32:57.078 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43064.log.gz 2024-04-26T20:32:57.079 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-04-26T20:32:57.079 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-04-26T20:32:57.080 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48226.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T20:32:57.080 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-04-26T20:32:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46577.log 2024-04-26T20:32:57.081 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-04-26T20:32:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41383.log 2024-04-26T20:32:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46577.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46577.log.gz 2024-04-26T20:32:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104823.log 2024-04-26T20:32:57.084 INFO:teuthology.orchestra.run.smithi043.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T20:32:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41719.log 2024-04-26T20:32:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41383.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41383.log.gz 2024-04-26T20:32:57.086 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44491.log 2024-04-26T20:32:57.087 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104823.log.gz 2024-04-26T20:32:57.087 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33678.log 2024-04-26T20:32:57.088 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41719.log.gz 2024-04-26T20:32:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44491.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T20:32:57.089 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44491.log.gz 2024-04-26T20:32:57.090 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39853.log 2024-04-26T20:32:57.090 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33678.log.gz 2024-04-26T20:32:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T20:32:57.093 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-04-26T20:32:57.093 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39853.log.gz 2024-04-26T20:32:57.094 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44675.log 2024-04-26T20:32:57.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39271.log 2024-04-26T20:32:57.096 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-04-26T20:32:57.096 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40514.log 2024-04-26T20:32:57.097 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44675.log.gz 2024-04-26T20:32:57.098 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39271.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T20:32:57.098 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39271.log.gz 2024-04-26T20:32:57.099 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-04-26T20:32:57.100 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40514.log.gz 2024-04-26T20:32:57.100 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39087.log 2024-04-26T20:32:57.102 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-04-26T20:32:57.102 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-04-26T20:32:57.103 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47039.log 2024-04-26T20:32:57.104 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39087.log.gz 2024-04-26T20:32:57.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2024-04-26T20:32:57.105 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-04-26T20:32:57.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47039.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47039.log.gz 2024-04-26T20:32:57.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.45047.log: --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-04-26T20:32:57.108 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2024-04-26T20:32:57.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105756.log 2024-04-26T20:32:57.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105977.log 2024-04-26T20:32:57.110 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-04-26T20:32:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105756.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105756.log.gz 2024-04-26T20:32:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105977.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105977.log.gz 2024-04-26T20:32:57.113 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-04-26T20:32:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-04-26T20:32:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43275.log 2024-04-26T20:32:57.116 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-04-26T20:32:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37843.log 2024-04-26T20:32:57.118 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-04-26T20:32:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40787.log 2024-04-26T20:32:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43275.log: 88.3%/var/log/ceph/ceph-client.admin.37843.log: -- replaced with /var/log/ceph/ceph-client.admin.43275.log.gz 2024-04-26T20:32:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37843.log.gz 2024-04-26T20:32:57.120 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-04-26T20:32:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40787.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40787.log.gz 2024-04-26T20:32:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48016.log 2024-04-26T20:32:57.123 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-04-26T20:32:57.124 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105304.log 2024-04-26T20:32:57.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-04-26T20:32:57.126 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48016.log.gz 2024-04-26T20:32:57.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-04-26T20:32:57.127 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105304.log.gz 2024-04-26T20:32:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45809.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-04-26T20:32:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.37989.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-04-26T20:32:57.130 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-04-26T20:32:57.130 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-04-26T20:32:57.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105628.log 2024-04-26T20:32:57.132 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2024-04-26T20:32:57.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44060.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-04-26T20:32:57.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106187.log 2024-04-26T20:32:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105628.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105628.log.gz 2024-04-26T20:32:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2024-04-26T20:32:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105117.log 2024-04-26T20:32:57.138 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106187.log.gz 2024-04-26T20:32:57.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41425.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2024-04-26T20:32:57.139 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.105117.log: --verbose -- /var/log/ceph/ceph-client.admin.105075.log 2024-04-26T20:32:57.139 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105117.log.gz 2024-04-26T20:32:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105934.log 2024-04-26T20:32:57.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67100.log 2024-04-26T20:32:57.142 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105075.log.gz 2024-04-26T20:32:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39685.log 2024-04-26T20:32:57.143 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105934.log.gz 2024-04-26T20:32:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.67100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-04-26T20:32:57.144 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67100.log.gz 2024-04-26T20:32:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39685.log.gz 2024-04-26T20:32:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106061.log 2024-04-26T20:32:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40501.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2024-04-26T20:32:57.148 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-04-26T20:32:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-04-26T20:32:57.149 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106061.log.gz 2024-04-26T20:32:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41215.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-04-26T20:32:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T20:32:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43892.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-04-26T20:32:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48286.log 2024-04-26T20:32:57.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46283.log 2024-04-26T20:32:57.155 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T20:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48286.log.gz 2024-04-26T20:32:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2024-04-26T20:32:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46283.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46283.log.gz 2024-04-26T20:32:57.157 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-04-26T20:32:57.158 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45089.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2024-04-26T20:32:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105201.log 2024-04-26T20:32:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47369.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-04-26T20:32:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47242.log 2024-04-26T20:32:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105201.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105201.log.gz 2024-04-26T20:32:57.163 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-04-26T20:32:57.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47242.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47242.log.gz 2024-04-26T20:32:57.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26071.log 2024-04-26T20:32:57.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45971.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-04-26T20:32:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106103.log/var/log/ceph/ceph.tmp-client.admin.26071.log: 2024-04-26T20:32:57.167 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26071.log.gz 2024-04-26T20:32:57.169 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47161.log 2024-04-26T20:32:57.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106103.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106103.log.gz 2024-04-26T20:32:57.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.47161.log: --verbose -- /var/log/ceph/ceph-client.admin.37947.log 2024-04-26T20:32:57.171 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47161.log.gz 2024-04-26T20:32:57.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39313.log 2024-04-26T20:32:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37947.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37947.log.gz 2024-04-26T20:32:57.175 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.39313.log: --verbose -- /var/log/ceph/ceph-client.admin.41761.log 2024-04-26T20:32:57.175 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39313.log.gz 2024-04-26T20:32:57.177 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39397.log 2024-04-26T20:32:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41761.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41761.log.gz 2024-04-26T20:32:57.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39397.log.gz 2024-04-26T20:32:57.180 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:57.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2024-04-26T20:32:57.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46787.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-26T20:32:57.183 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40504.log 2024-04-26T20:32:57.184 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T20:32:57.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44879.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2024-04-26T20:32:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40504.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40504.log.gz 2024-04-26T20:32:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-04-26T20:32:57.188 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43320.log 2024-04-26T20:32:57.189 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43850.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-04-26T20:32:57.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105892.log 2024-04-26T20:32:57.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43320.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43320.log.gz 2024-04-26T20:32:57.191 INFO:teuthology.orchestra.run.smithi043.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46367.log 2024-04-26T20:32:57.192 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T20:32:57.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105892.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105892.log.gz 2024-04-26T20:32:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46367.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46367.log.gz 2024-04-26T20:32:57.195 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40177.log 2024-04-26T20:32:57.196 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-04-26T20:32:57.198 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40177.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40177.log.gz 2024-04-26T20:32:57.198 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40505.log 2024-04-26T20:32:57.199 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44963.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-04-26T20:32:57.199 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37707.log 2024-04-26T20:32:57.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2024-04-26T20:32:57.201 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40505.log.gz 2024-04-26T20:32:57.202 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37707.log.gz 2024-04-26T20:32:57.203 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40512.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2024-04-26T20:32:57.204 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39171.log 2024-04-26T20:32:57.205 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-04-26T20:32:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39171.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39171.log.gz 2024-04-26T20:32:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47844.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-04-26T20:32:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44323.log 2024-04-26T20:32:57.210 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43976.log 2024-04-26T20:32:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-04-26T20:32:57.211 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44323.log.gz 2024-04-26T20:32:57.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43976.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43976.log.gz 2024-04-26T20:32:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47327.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-04-26T20:32:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46409.log 2024-04-26T20:32:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2024-04-26T20:32:57.217 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42414.log 2024-04-26T20:32:57.217 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46409.log.gz 2024-04-26T20:32:57.219 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47411.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2024-04-26T20:32:57.220 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42414.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2024-04-26T20:32:57.220 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38117.log 2024-04-26T20:32:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105712.log 2024-04-26T20:32:57.223 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38201.log 2024-04-26T20:32:57.223 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38117.log.gz 2024-04-26T20:32:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105712.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105712.log.gz 2024-04-26T20:32:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38201.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38201.log.gz 2024-04-26T20:32:57.226 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45683.log 2024-04-26T20:32:57.228 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44759.log 2024-04-26T20:32:57.229 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40500.log 2024-04-26T20:32:57.229 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45683.log.gz 2024-04-26T20:32:57.231 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44759.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44759.log.gz 2024-04-26T20:32:57.232 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40500.log.gz 2024-04-26T20:32:57.232 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-04-26T20:32:57.234 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-04-26T20:32:57.235 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-04-26T20:32:57.235 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-26T20:32:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47537.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-04-26T20:32:57.238 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47886.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-04-26T20:32:57.238 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44186.log 2024-04-26T20:32:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-04-26T20:32:57.242 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106313.log 2024-04-26T20:32:57.242 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44186.log.gz 2024-04-26T20:32:57.242 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46139.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-04-26T20:32:57.244 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106313.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106313.log.gz 2024-04-26T20:32:57.244 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46913.log 2024-04-26T20:32:57.245 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47579.log 2024-04-26T20:32:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45173.log 2024-04-26T20:32:57.247 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46913.log.gz 2024-04-26T20:32:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47579.log.gz 2024-04-26T20:32:57.249 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45173.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45173.log.gz 2024-04-26T20:32:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106397.log 2024-04-26T20:32:57.251 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40846.log 2024-04-26T20:32:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48328.log 2024-04-26T20:32:57.253 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106397.log.gz 2024-04-26T20:32:57.254 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39601.log 2024-04-26T20:32:57.254 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40846.log.gz 2024-04-26T20:32:57.256 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2024-04-26T20:32:57.256 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48328.log.gz 2024-04-26T20:32:57.257 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39601.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-04-26T20:32:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44591.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2024-04-26T20:32:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105798.log 2024-04-26T20:32:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39045.log 2024-04-26T20:32:57.261 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105798.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.105798.log.gz 2024-04-26T20:32:57.262 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-04-26T20:32:57.263 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39045.log: gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.44407.log -- replaced with /var/log/ceph/ceph-client.admin.39045.log.gz 2024-04-26T20:32:57.264 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:57.265 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46955.log 2024-04-26T20:32:57.265 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-04-26T20:32:57.266 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44407.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44407.log.gz 2024-04-26T20:32:57.267 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.46955.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39913.log 2024-04-26T20:32:57.268 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46955.log.gz 2024-04-26T20:32:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106439.log 2024-04-26T20:32:57.270 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39913.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45257.log 2024-04-26T20:32:57.271 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39913.log.gz 2024-04-26T20:32:57.272 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40850.log 2024-04-26T20:32:57.272 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106439.log.gz 2024-04-26T20:32:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45257.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45257.log.gz 2024-04-26T20:32:57.274 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106355.log 2024-04-26T20:32:57.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40850.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40850.log.gz 2024-04-26T20:32:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-04-26T20:32:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-04-26T20:32:57.277 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106355.log.gz 2024-04-26T20:32:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43369.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-04-26T20:32:57.279 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.40174.log 2024-04-26T20:32:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45005.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-04-26T20:32:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38393.log 2024-04-26T20:32:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104781.log 2024-04-26T20:32:57.281 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2024-04-26T20:32:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38393.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38393.log.gz 2024-04-26T20:32:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104781.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.104781.log.gz /var/log/ceph/ceph-client.admin.40515.log 2024-04-26T20:32:57.284 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:57.286 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40851.log 2024-04-26T20:32:57.287 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40515.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-04-26T20:32:57.287 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-04-26T20:32:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-04-26T20:32:57.289 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40851.log.gz 2024-04-26T20:32:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40777.log 2024-04-26T20:32:57.290 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-04-26T20:32:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45457.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-04-26T20:32:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40179.log 2024-04-26T20:32:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42808.log 2024-04-26T20:32:57.293 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40777.log.gz 2024-04-26T20:32:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-04-26T20:32:57.294 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40179.log.gz 2024-04-26T20:32:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.42808.log: /var/log/ceph/ceph-client.admin.43934.log: 92.4% -- replaced with /var/log/ceph/ceph-client.2.42808.log.gz 2024-04-26T20:32:57.297 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-04-26T20:32:57.298 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-04-26T20:32:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48370.log 2024-04-26T20:32:57.301 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46829.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106500.log 2024-04-26T20:32:57.301 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-04-26T20:32:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47663.log 2024-04-26T20:32:57.302 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48370.log.gz 2024-04-26T20:32:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106500.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106500.log.gz 2024-04-26T20:32:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46451.log 2024-04-26T20:32:57.305 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47663.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47663.log.gz 2024-04-26T20:32:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-04-26T20:32:57.307 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46451.log: gzip -5 --verbose -- 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46451.log.gz 2024-04-26T20:32:57.307 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.46097.log 2024-04-26T20:32:57.308 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43724.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-04-26T20:32:57.309 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38830.log 2024-04-26T20:32:57.310 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-04-26T20:32:57.310 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-04-26T20:32:57.312 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106019.log 2024-04-26T20:32:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39643.log: 92.5% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-04-26T20:32:57.314 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38830.log.gz 2024-04-26T20:32:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-04-26T20:32:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106019.log.gz 2024-04-26T20:32:57.316 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104949.log 2024-04-26T20:32:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41677.log 2024-04-26T20:32:57.317 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-04-26T20:32:57.318 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104949.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104949.log.gz 2024-04-26T20:32:57.320 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41677.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41677.log.gz 2024-04-26T20:32:57.320 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-04-26T20:32:57.321 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47932.log 2024-04-26T20:32:57.323 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67599.log 2024-04-26T20:32:57.323 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-04-26T20:32:57.324 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47932.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47932.log.gz 2024-04-26T20:32:57.325 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37663.log 2024-04-26T20:32:57.326 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.67599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105033.log 2024-04-26T20:32:57.326 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67599.log.gz 2024-04-26T20:32:57.327 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38159.log 2024-04-26T20:32:57.327 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37663.log.gz 2024-04-26T20:32:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105261.log 2024-04-26T20:32:57.329 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105033.log.gz 2024-04-26T20:32:57.330 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105670.log 2024-04-26T20:32:57.330 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38159.log.gz 2024-04-26T20:32:57.331 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105261.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T20:32:57.331 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105261.log.gz 2024-04-26T20:32:57.333 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105670.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105670.log.gz 2024-04-26T20:32:57.333 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.40743.log 2024-04-26T20:32:57.335 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-04-26T20:32:57.336 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40743.log.gz 2024-04-26T20:32:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-04-26T20:32:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42309.log 2024-04-26T20:32:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-04-26T20:32:57.343 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.42309.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.42309.log.gz 2024-04-26T20:32:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-04-26T20:32:57.345 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47974.log 2024-04-26T20:32:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38327.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-04-26T20:32:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45541.log 2024-04-26T20:32:57.349 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47974.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47974.log.gz 2024-04-26T20:32:57.349 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105848.log 2024-04-26T20:32:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45541.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45541.log.gz 2024-04-26T20:32:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-04-26T20:32:57.352 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105848.log.gz 2024-04-26T20:32:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38285.log 2024-04-26T20:32:57.355 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-04-26T20:32:57.355 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104865.log 2024-04-26T20:32:57.356 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38285.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38285.log.gz 2024-04-26T20:32:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-04-26T20:32:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104865.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104865.log.gz 2024-04-26T20:32:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43472.log: /var/log/ceph/ceph-client.admin.104907.log 2024-04-26T20:32:57.360 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-04-26T20:32:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40849.log 2024-04-26T20:32:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.104907.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104907.log.gz 2024-04-26T20:32:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41509.log 2024-04-26T20:32:57.364 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40849.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40849.log.gz 2024-04-26T20:32:57.365 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-04-26T20:32:57.366 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41509.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41509.log.gz 2024-04-26T20:32:57.367 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41173.log 2024-04-26T20:32:57.368 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43640.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-04-26T20:32:57.369 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39811.log/var/log/ceph/ceph-client.admin.41173.log: 2024-04-26T20:32:57.370 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41173.log.gz 2024-04-26T20:32:57.371 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-04-26T20:32:57.372 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39811.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39811.log.gz 2024-04-26T20:32:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43682.log: /var/log/ceph/ceph-client.admin.106271.log 2024-04-26T20:32:57.374 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-04-26T20:32:57.375 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37905.log 2024-04-26T20:32:57.377 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.106271.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106271.log.gz 2024-04-26T20:32:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39129.log 2024-04-26T20:32:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37905.log.gz 2024-04-26T20:32:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-04-26T20:32:57.380 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39129.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39129.log.gz 2024-04-26T20:32:57.381 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48184.log 2024-04-26T20:32:57.382 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46055.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-04-26T20:32:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-04-26T20:32:57.384 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48184.log.gz 2024-04-26T20:32:57.385 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44921.log 2024-04-26T20:32:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46535.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-04-26T20:32:57.388 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2024-04-26T20:32:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44921.log.gz 2024-04-26T20:32:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39213.log 2024-04-26T20:32:57.391 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41131.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2024-04-26T20:32:57.392 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-04-26T20:32:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39213.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39213.log.gz 2024-04-26T20:32:57.394 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39727.log 2024-04-26T20:32:57.395 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40844.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-04-26T20:32:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T20:32:57.396 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40167.log 2024-04-26T20:32:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39727.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39727.log.gz 2024-04-26T20:32:57.397 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-04-26T20:32:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45499.log 2024-04-26T20:32:57.399 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40167.log.gz 2024-04-26T20:32:57.400 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47284.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-04-26T20:32:57.401 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105388.log 2024-04-26T20:32:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45499.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45499.log.gz 2024-04-26T20:32:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45767.log 2024-04-26T20:32:57.403 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.105388.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105388.log.gz 2024-04-26T20:32:57.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45767.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45767.log.gz 2024-04-26T20:32:57.553 INFO:teuthology.orchestra.run.smithi043.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T20:32:57.730 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T20:32:57.753 INFO:teuthology.orchestra.run.smithi164.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T20:32:57.820 INFO:teuthology.orchestra.run.smithi164.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T20:32:57.869 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T20:32:57.878 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T20:32:57.883 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-26T20:32:57.883 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.934s 2024-04-26T20:32:57.883 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m3.366s 2024-04-26T20:32:57.883 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.362s 2024-04-26T20:32:57.957 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T20:32:58.050 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T20:32:58.076 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T20:32:58.081 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-26T20:32:58.081 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m1.131s 2024-04-26T20:32:58.081 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m4.040s 2024-04-26T20:32:58.081 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m1.083s 2024-04-26T20:32:58.082 INFO:tasks.ceph:Archiving logs... 2024-04-26T20:32:58.083 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi043/log 2024-04-26T20:32:58.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:32:58.854 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi164/log 2024-04-26T20:32:58.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:32:59.403 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T20:32:59.412 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T20:32:59.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:32:59.420 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:32:59.575 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T20:32:59.584 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T20:32:59.584 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:32:59.594 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:32:59.643 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:32:59.649 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +1356us[+1359us] +/- 59ms 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -1007us[-1004us] +/- 77ms 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -956us[ -956us] +/- 89ms 2024-04-26T20:32:59.653 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1353us[+1367us] +/- 59ms 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -1007us[ -994us] +/- 77ms 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -970us[ -957us] +/- 89ms 2024-04-26T20:32:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:32:59.660 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T20:32:59.670 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T20:32:59.671 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T20:32:59.680 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:32:59.897 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:33:00.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:33:00.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:33:00.182 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:33:00.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:33:00.277 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T20:33:00.287 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T20:33:00.297 INFO:teuthology.task.internal:Duration was 953.506667 seconds 2024-04-26T20:33:00.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T20:33:00.308 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T20:33:00.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:33:00.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:33:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:33:00.451 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:33:00.793 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T20:33:00.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-04-26T20:33:00.794 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:33:00.857 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-26T20:33:00.858 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:33:00.934 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T20:33:00.934 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:33:00.942 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:33:01.031 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T20:33:01.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:33:01.040 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:33:01.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T20:33:01.235 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T20:33:01.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:33:01.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:33:01.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T20:33:01.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:33:01.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:33:01.525 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-26T20:33:01.527 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-04-26T20:33:01.576 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:33:01.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:33:01.628 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:33:01.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:33:01.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T20:33:01.694 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T20:33:01.695 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi043 2024-04-26T20:33:01.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:33:01.802 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164/remote/smithi164 2024-04-26T20:33:01.803 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:33:01.897 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T20:33:01.897 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:33:01.906 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:33:01.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T20:33:01.972 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T20:33:01.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T20:33:01.982 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T20:33:01.982 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:33:01.991 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:33:02.042 INFO:teuthology.orchestra.run.smithi043.stdout: 393229 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 26 20:33 /home/ubuntu/cephtest 2024-04-26T20:33:02.042 INFO:teuthology.orchestra.run.smithi043.stdout: 527347 4 d--------- 2 ubuntu ubuntu 4096 Apr 26 20:28 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:33:02.048 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-26T20:33:02.048 INFO:teuthology.orchestra.run.smithi164.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:33 /home/ubuntu/cephtest 2024-04-26T20:33:02.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:33:02.049 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-26T20:33:02.050 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi043 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-26T20:33:02.051 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T20:33:02.061 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T20:33:02.074 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi043 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-26T20:33:02.074 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T20:33:02.099 DEBUG:teuthology.nuke:shortname: smithi043 2024-04-26T20:33:02.100 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:33:02.122 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:07:35.582378', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOu3G4ORMpFFzdyVlekiwhZkIWNMgClxKe2xZr7nLLokggQyFpD3KjXtfiCzRw12iWT0SOiI6I435VB5xx43s8k='} 2024-04-26T20:33:02.125 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-26T20:33:02.125 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:33:02.149 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675164', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:07:35.580888', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPeG7ialqsj8nIaFb16MqFvM/lqPQZj8cA0W+i2UkjT76rNJuy6/e7d9HlvZuJQyWgpzLSEQ9Ym86BwGF805KHk='} 2024-04-26T20:33:02.174 INFO:teuthology.orchestra.console.smithi043:Power off 2024-04-26T20:33:02.175 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:33:02.197 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-04-26T20:33:02.197 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:02.216 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-26T20:33:02.216 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:33:02.226 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-26T20:33:02.240 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-26T20:33:02.240 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:02.269 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-26T20:33:06.228 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:06.270 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:06.339 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-26T20:33:06.380 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-26T20:33:10.341 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:10.381 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:10.451 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-04-26T20:33:10.491 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-26T20:33:10.553 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-04-26T20:33:14.495 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:33:14.605 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-26T20:33:14.706 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-26T20:33:14.811 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 953.5066668987274 failure_reason: 'Command failed on smithi043 with status 1: ''find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest''' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology status: fail success: false 2024-04-26T20:33:14.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:33:14.896 INFO:teuthology.run:FAIL